date:20110303


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001974#comment-13001974
 ] 

Gary Dusbabek commented on CASSANDRA-2124:
--

bq. Should we hold a decoder instance with Connection or it should be a static 
call? 
I figured it would be easier to maintain the current schema info if the decoder 
weren't static.  That way when a new connection is established, new schema is 
loaded.  Otherwise, we'd need to figure out a way to update the 
static/singleton.

bq. Passing Decoder instance to CassandraResultSet is required, why?
Because that is where the decoding is actually done.

bq. Planning for ResultSetMetadata API? 
Yes. Baby steps for now.

bq. CassandraResultSet does not need column name decoding(only value decoding), 
i think.
To use the ResultSet.getXXX(String) methods, we need a way to map column names 
to strings.

 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v1-0001-first-pass-at-column-decoding.txt, 
 v1-0002-clean-up-JdbcDriverTest.txt, 
 v1-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v1-0004-more-comments-in-ColumnDecoder.txt


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL

2011-03-03 Thread Vivek Mishra (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001977#comment-13001977
 ] 

Vivek Mishra commented on CASSANDRA-2124:
-

| I figured it would be easier to maintain the current schema info if the 
decoder weren't static. That way when a new connection is established, new 
schema is loaded. Otherwise, we'd need to figure out a way to update the 
static/singleton.

Ok. Now i understand, reason to hold this with connection instance is make it 
reusable with other future drivers, if reqd.

|To use the ResultSet.getXXX(String) methods, we need a way to map column names 
to strings.

For this, either we need to introduce a wrapper or Ascii, BytesType, UUID etc. 
classes need to provide a way for toString() calls.

Let me know, if i can take a look on something which pops up on your mind.




 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v1-0001-first-pass-at-column-decoding.txt, 
 v1-0002-clean-up-JdbcDriverTest.txt, 
 v1-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v1-0004-more-comments-in-ColumnDecoder.txt


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2233) Add unified UUIDType


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13001998#comment-13001998
 ] 

Jonathan Ellis commented on CASSANDRA-2233:
---

bq. I'm not sure how useful the natural order of the other UUID versions is

Agreed.

bq. don't access ByteBuffer.array() directly without asking if 
ByteBuffer.hasArray() first

Yes, we have a bunch of methods in ByteBufferUtil that will use array if it 
exists otherwise will either duplicate and use relative methods or use 
byte-at-a-time get.

bq. write a unit test.

+1, mostly it will just need to adapt what we have in TimeUUIDTypeTeset

 Add unified UUIDType
 

 Key: CASSANDRA-2233
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2233
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: UUIDType.java


 Unified UUIDType comparator, compares as time-based if both UUIDs are 
 time-based, otherwise uses byte comparison.  Based on code from the current 
 LexicalUUIDType and TimeUUIDType comparers, so performance and behavior 
 should be consistent and compatible.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek reassigned CASSANDRA-2262:


Assignee: Gary Dusbabek

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v0-0001-test-shows-no-roundtrip-in-BytesType.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2264) Restore From Snapshot eg: nodetool restore [filename] | ([keyspace] [columnfamily])


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002030#comment-13002030
 ] 

Jonathan Ellis commented on CASSANDRA-2264:
---

Why do we need to store additional metadata?  ISTM that restore from snapshot 
should just be make the snapshotted sstables live.

 Restore From Snapshot eg: nodetool restore  [filename] | ([keyspace] 
 [columnfamily]) 
 -

 Key: CASSANDRA-2264
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2264
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.2
Reporter: Benjamin Coverston

 Store additional metadata in the SSTable including:
 generated_by: (From memtable, compaction, or cleanup)
 ancestors: A list of SSTableNames|MD5sum
 When executed it will copy the ancestors from the snapshot directory to the 
 data directory.
 If given Keyspace and ColumnFamily arguments it will attempt to restore all 
 SSTables in the Keyspace/Columnfamily on that node.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076662 - in /cassandra/branches/cassandra-0.7: src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.jav

Author: jbellis
Date: Thu Mar  3 16:06:16 2011
New Revision: 1076662

URL: http://svn.apache.org/viewvc?rev=1076662view=rev
Log:
throw EOFException when seeking past EOF in read-only mode
patch by Pavel Yaskevich; reviewed by tjake and jbellis for CASSANDRA-2256

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java?rev=1076662r1=1076661r2=1076662view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java
 Thu Mar  3 16:06:16 2011
@@ -17,6 +17,7 @@
  */
 package org.apache.cassandra.io.util;
 
+import java.io.EOFException;
 import java.io.File;
 import java.io.IOException;
 import java.io.RandomAccessFile;
@@ -291,6 +292,9 @@ public class BufferedRandomAccessFile ex
 if (buffer == null)
 throw new ClosedChannelException();
 
+if (isReadOnly())
+throw new IOException(Unable to write: file is in the read-only 
mode.);
+
 while (length  0)
 {
 int n = writeAtMost(buff, offset, length);
@@ -301,6 +305,11 @@ public class BufferedRandomAccessFile ex
 }
 }
 
+private boolean isReadOnly()
+{
+return fileLength != -1;
+}
+
 /*
  * Write at most length bytes from b starting at position offset, and
  * return the number of bytes written. caller is responsible for setting
@@ -328,6 +337,9 @@ public class BufferedRandomAccessFile ex
 if (newPosition  0)
 throw new IllegalArgumentException(new position should not be 
negative);
 
+if (isReadOnly()  newPosition  fileLength)
+throw new EOFException(unable to seek past the end of the file in 
read-only mode.);
+
 current = newPosition;
 
 if (newPosition = bufferOffset + validBufferBytes || newPosition  
bufferOffset)

Modified: 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java?rev=1076662r1=1076661r2=1076662view=diff
==
--- 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/io/util/BufferedRandomAccessFileTest.java
 Thu Mar  3 16:06:16 2011
@@ -521,6 +521,107 @@ public class BufferedRandomAccessFileTes
 file.bytesPastMark();
 }
 
+@Test
+public void testReadOnly() throws IOException
+{
+BufferedRandomAccessFile file = createTempFile(brafReadOnlyTest);
+
+byte[] data = new byte[20];
+for (int i = 0; i  data.length; i++)
+data[i] = 'c';
+
+file.write(data);
+file.sync(); // flushing file to the disk
+
+// read-only copy of the file, with fixed file length
+final BufferedRandomAccessFile copy = new 
BufferedRandomAccessFile(file.getPath(), r);
+
+copy.seek(copy.length());
+assertTrue(copy.bytesRemaining() == 0  copy.isEOF());
+
+// can't seek past the end of the file for read-only files
+expectEOF(new CallableObject()
+{
+public Object call() throws IOException
+{
+copy.seek(copy.length() + 1);
+return null;
+}
+});
+
+/* Any write() call should fail */
+expectException(new CallableObject()
+{
+public Object call() throws IOException
+{
+copy.write(1);
+return null;
+}
+}, IOException.class);
+
+expectException(new CallableObject()
+{
+public Object call() throws IOException
+{
+copy.write(new byte[1]);
+return null;
+}
+}, IOException.class);
+
+expectException(new CallableObject()
+{
+public Object call() throws IOException
+{
+copy.write(new byte[3], 0, 2);
+return null;
+}
+}, IOException.class);
+
+copy.seek(0);
+copy.skipBytes(5);
+
+assertEquals(copy.bytesRemaining(), 15);
+assertEquals(copy.getFilePointer(), 5);
+

[jira] Commented: (CASSANDRA-2256) BRAF assertion error


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002036#comment-13002036
 ] 

Jonathan Ellis commented on CASSANDRA-2256:
---

committed with additional test that writing past EOF actually works

 BRAF assertion error
 

 Key: CASSANDRA-2256
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2256
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.3
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.7.4

 Attachments: CASSANDRA-2256-v2.patch, CASSANDRA-2256.patch


 While investigating CASSANDRA-2240 I ran into this:
 {noformat}
 java.lang.AssertionError
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.read(BufferedRandomAccessFile.java\
 :230)
 at java.io.RandomAccessFile.readByte(RandomAccessFile.java:589)
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:273)
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:284)
 at 
 org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:539)
 {noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002037#comment-13002037
 ] 

Jonathan Ellis commented on CASSANDRA-2228:
---

It looks like we're doing Migration stuff before Gossiper.start is called.

One possible solution: initialize Gossiper.localEndpoint in constructor instead 
of in start.

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2) If I restart then, I'm getting an

[jira] Commented: (CASSANDRA-2256) BRAF assertion error

2011-03-03 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002049#comment-13002049
 ] 

Hudson commented on CASSANDRA-2256:
---

Integrated in Cassandra-0.7 #341 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/341/])
throw EOFException when seeking past EOF in read-only mode
patch by Pavel Yaskevich; reviewed by tjake and jbellis for CASSANDRA-2256


 BRAF assertion error
 

 Key: CASSANDRA-2256
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2256
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.3
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.7.4

 Attachments: CASSANDRA-2256-v2.patch, CASSANDRA-2256.patch


 While investigating CASSANDRA-2240 I ran into this:
 {noformat}
 java.lang.AssertionError
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.read(BufferedRandomAccessFile.java\
 :230)
 at java.io.RandomAccessFile.readByte(RandomAccessFile.java:589)
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:273)
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:284)
 at 
 org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:539)
 {noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2244) repair doesn't handle secondary indexes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2244:
--

Attachment: 2244.txt

The don't bother flushing dropped CFs check was unaware that index CFs are 
never part of the global CF metadata.

(I think that check is race-prone to begin with, so we should probably drop it 
and do something more robust, but for now it's better than nothing -- it will 
*usually* prevent creating new sstables for dropped CFs.)

 repair doesn't handle secondary indexes
 ---

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4

 Attachments: 2244.txt


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-2231:


Attachment: 0001-Add-compositeType-and-DynamicCompositeType.patch

bq. (1) having validation only require that the components, if they're 
provided, are of the correct type (looking at Sylvain's code, the compare 
function should work, but the validation wouldn't)

That's a bug. The last test of validate() should be negated. The only goal was 
forbid having leading bytes after all the comparator have been used (because 
compare makes the assumption that there will not have bytes without a proper 
comparator for them).

However it is clear that we should allow a key having less component than 
comparator (btw, the unit test in my patch was actually doing it).

This is fixed in the patch I'm attaching.

bq. My other concern is that one of the things that got stripped out was the 
MATCH_MINIMUM, MATCH_MAXIMUM feature, which made implementing 
greater-than-equals, less-than-equals, etc. in ranges much easier

I'm not familiar enough with your code to be sure what those were doing 
exactly, but I suppose that the goal is that, if for instance you have the 
following column names:
{noformat}
test:bar:42
test:foo:24
test:foo:42
test:foobar
{noformat}
then you want to be able to query anything that starts with the two component 
{{test}} and {{foo}} (that is {{test:foo:24}} and {{test:foo:42}} in my 
example).

Greater-than is already doable in my previous patch (up to the bug in 
validation). For the less-than part, I agree that it is nice to be able to do 
it easily. In my new patch, I add a leading byte to each component, whose 
purpose is to always be 0, except for lesser-than query. That way, you can do 
the query above easily. The price is a slightly more complicated encoding but I 
think it's totally worth it.


bq. While i agree in theory, this is why thrift exists right?

I don't know thrift as well as you do, but I would think that changing the type 
of column name would be a massive compatibility breaker. And I really don't see 
a not crappy way to do it. Well, actually I do, but it consists in revamping 
the API completely, ditching the super columns in favor of composite column 
name. Don't get me wrong, I'm 100% in, but it's well beyond this particular 
ticket and I think that as long as we have a generic and simple enough encoding 
for those composite type, I believe it's ok to include this as is and just have 
a decent description of the encoding for client to use. We can still branch 
that later on on thrift anyway.

bq. (2) creating a DynamicType that could be used for dynamic types

I do agree with you that it could be useful and I understand the use case. I 
also agree that whatever we do, we should make it clear that it is not beginner 
stuff and that you shouldn't use a dynamic composite type unless you really 
need it (since the possibility of shooting yourself in the foot if you don't 
know what you do is fairly high).  All this being said, I took a stab at a 
DynamicCompositeType and I think the result really makes sense.

The main idea is really just to modify my static CompositeType so that each 
component starts with the name of the comparator class (that way, this is again 
completely generic, custom comparator can be used without problems).  There is 
then 2 optimisations to make it a reasonable solution:
  # FBUtilities.getComparator() uses a cache, to avoid the cost of 
introspection.
  # There is a notion of alias. They are user defined and are local to each 
DynamicCompositeType (cf. test/conf/cassandra.yaml in the patch). An alias is a 
simple (printable) ascii character (there is around 100 possible alias which 
should be aplenty). You can then use those alias in the encoded column name, in 
which case you only need 2 bytes to encode the comparator, making it fairly 
compact.
The encoded format is very close to the one of the static CompositeType, even 
though it is a tiny bit more complex due to the aliases handling. But given 
that dynamic composite type is not meant for the faint of heart, that should be 
fine.

I'm attaching a new patch (against 0.7 branch) with both the static and dynamic 
composite type. The bulk of the code is shared between them. This also include 
a fair amount of unit tests.


 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch,

[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2244:
--

Description: The repaired node neither receives indexes from the replicas, 
nor does it generate them afterwards.  The same bug prevents generation of new 
indexes against existing data.  (was: The repaired node neither receives 
indexes from the replicas, nor does it generate them afterwards.)
Summary: secondary indexes aren't created on pre-existing or streamed 
data  (was: repair doesn't handle secondary indexes)

 secondary indexes aren't created on pre-existing or streamed data
 -

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4

 Attachments: 2244.txt


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.  The same bug prevents generation of new indexes 
 against existing data.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2244:
--

Attachment: (was: 2244.txt)

 secondary indexes aren't created on pre-existing or streamed data
 -

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.  The same bug prevents generation of new indexes 
 against existing data.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2244:
--

Attachment: 2244-v2.txt

patch adds test that catches the problem, and adds fix to CFS.

 secondary indexes aren't created on pre-existing or streamed data
 -

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4

 Attachments: 2244-v2.txt


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.  The same bug prevents generation of new indexes 
 against existing data.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: (was: v0-0001-test-shows-no-roundtrip-in-BytesType.txt)

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
v1-0003-compose-method-for-AbstractTypes.txt
v1-0002-BytesType.fromString-expects-a-hex-string.txt
v1-0001-test-shows-no-roundtrip-in-BytesType.txt

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002073#comment-13002073
 ] 

Jonathan Ellis commented on CASSANDRA-2244:
---

Gary points out that migration does acquire the flushlock.

 secondary indexes aren't created on pre-existing or streamed data
 -

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4

 Attachments: 2244-v2.txt


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.  The same bug prevents generation of new indexes 
 against existing data.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002080#comment-13002080
 ] 

Gary Dusbabek commented on CASSANDRA-2228:
--

CHM throws NPE when the key is null.  To me, then, the bug is that 
FBU.getLocalAddress() is somehow returning null.  

I checked yesterday: even if local_address is unspecified, it ends up returning 
localhost.

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2)

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2262:
--

Reviewer: slebresne

I take it compose() is give me a Java object that this value represents?

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002084#comment-13002084
 ] 

Jonathan Ellis commented on CASSANDRA-2228:
---

My point was that we can call put before initializing localEndpoint_ to 
FBU.gLA, so it's naturally going to be null.

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2) If I restart then, I'm getting an Nullpointer exception. The OOM error 
 will only appear

[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002089#comment-13002089
 ] 

Sylvain Lebresne commented on CASSANDRA-2262:
-

I don't think that matter for the purpose of this ticket, but as a remark, 
IntegerType doesn't always roundtrip correctly. For example
{noformat}
@Test
public void testInteger2()
{
long v = 1;
ByteBuffer bb = ByteBuffer.allocate(8);
bb.putLong(v);
bb.flip();
String s = IntegerType.instance.getString(bb);
ByteBuffer internal = IntegerType.instance.fromString(s);
assert internal.getLong() == v;
}
{noformat}
fails (while it works on LongType) because of the internal use of BigInteger 
that pack the bytes as tightly as possible.

This is probably fine since
{noformat}
 IntegerType.getString(IntegerType.fromString(s)) == s
{noformat}
always holds. It's
{noformat}
 IntegerType.fromString(IntegerType.getString(b)) == b
{noformat}
that doesn't always hold.

Last remark, the tests of the first patch are testing the wrong equation.

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076699 - in /cassandra/branches/cassandra-0.7: src/java/org/apache/cassandra/db/ColumnFamilyStore.java test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java

2011-03-03 Thread brandonwilliams

Author: brandonwilliams
Date: Thu Mar  3 17:40:59 2011
New Revision: 1076699

URL: http://svn.apache.org/viewvc?rev=1076699view=rev
Log:
CFS correctly flushes index CFs.
Patch by jbellis, reviewed by brandonwilliams for CASSANDRA-2244

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1076699r1=1076698r2=1076699view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 Thu Mar  3 17:40:59 2011
@@ -685,8 +685,11 @@ public class ColumnFamilyStore implement
 {
 if (oldMemtable.isFrozen())
 return null;
-
-if (DatabaseDescriptor.getCFMetaData(metadata.cfId) == null)
+
+boolean isDropped = isIndex()
+  ? DatabaseDescriptor.getCFMetaData(table.name, 
getParentColumnfamily()) == null
+  : 
DatabaseDescriptor.getCFMetaData(metadata.cfId) == null;
+if (isDropped)
 return null; // column family was dropped. no point in 
flushing.
 
 assert memtable == oldMemtable;
@@ -2113,6 +2116,12 @@ public class ColumnFamilyStore implement
 return partitioner instanceof LocalPartitioner;
 }
 
+private String getParentColumnfamily()
+{
+assert isIndex();
+return columnFamily.split(\\.)[0];
+}
+
 /**
  * sets each cache's maximum capacity to 75% of its current size
  */

Modified: 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java?rev=1076699r1=1076698r2=1076699view=diff
==
--- 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
 Thu Mar  3 17:40:59 2011
@@ -307,7 +307,7 @@ public class ColumnFamilyStoreTest exten
 }
 
 @Test
-public void testIndexCreate() throws IOException, ConfigurationException, 
InterruptedException
+public void testIndexCreate() throws IOException, ConfigurationException, 
InterruptedException, ExecutionException
 {
 Table table = Table.open(Keyspace1);
 
@@ -324,6 +324,9 @@ public class ColumnFamilyStoreTest exten
 while (!SystemTable.isIndexBuilt(Keyspace1, 
cfs.getIndexedColumnFamilyStore(ByteBufferUtil.bytes(birthdate)).columnFamily))
 TimeUnit.MILLISECONDS.sleep(100);
 
+// we had a bug (CASSANDRA-2244) where index would get created but not 
flushed -- check for that  
+assert cfs.getIndexedColumnFamilyStore(cd.name).getSSTables().size()  
0;
+
 IndexExpression expr = new 
IndexExpression(ByteBufferUtil.bytes(birthdate), IndexOperator.EQ, 
ByteBufferUtil.bytes(1L));
 IndexClause clause = new IndexClause(Arrays.asList(expr), 
ByteBufferUtil.EMPTY_BYTE_BUFFER, 100);
 IFilter filter = new IdentityQueryFilter();
@@ -331,8 +334,7 @@ public class ColumnFamilyStoreTest exten
 Range range = new Range(p.getMinimumToken(), p.getMinimumToken());
 ListRow rows = table.getColumnFamilyStore(Indexed2).scan(clause, 
range, filter);
 assert rows.size() == 1 : StringUtils.join(rows, ,);
-String key = new 
String(rows.get(0).key.key.array(),rows.get(0).key.key.position(),rows.get(0).key.key.remaining());
 
-assert k1.equals( key );
+assertEquals(k1, ByteBufferUtil.string(rows.get(0).key.key));
 }
 
 @Test

svn commit: r1076700 - /cassandra/branches/cassandra-0.7/CHANGES.txt

2011-03-03 Thread brandonwilliams

Author: brandonwilliams
Date: Thu Mar  3 17:42:16 2011
New Revision: 1076700

URL: http://svn.apache.org/viewvc?rev=1076700view=rev
Log:
update CHANGES

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076700r1=1076699r2=1076700view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Mar  3 17:42:16 2011
@@ -1,5 +1,6 @@
 0.7.4
  * add nodetool join command (CASSANDRA-2160)
+ * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244)
 
 
 0.7.3

[jira] Commented: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data

2011-03-03 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002103#comment-13002103
 ] 

Hudson commented on CASSANDRA-2244:
---

Integrated in Cassandra-0.7 #342 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/342/])
CFS correctly flushes index CFs.
Patch by jbellis, reviewed by brandonwilliams for CASSANDRA-2244


 secondary indexes aren't created on pre-existing or streamed data
 -

 Key: CASSANDRA-2244
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2244
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Jonathan Ellis
 Fix For: 0.7.4

 Attachments: 2244-v2.txt


 The repaired node neither receives indexes from the replicas, nor does it 
 generate them afterwards.  The same bug prevents generation of new indexes 
 against existing data.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: 02473253(


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joaquin Casares updated CASSANDRA-1170:
---

Affects Version/s: 0.6.8

 java.lang.NumberFormatException: For input string: 02473253(
 --

 Key: CASSANDRA-1170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6.2, 0.6.8
 Environment: Cassandra 0.6.2
 Ubuntu 9.10 on EC2
 java version 1.6.0_0
 IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
 OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
Reporter: David King

 When trying to boot a node:
  INFO 11:05:16,117 Sampling index for 
 /cassandra/data/permacache/permacache-13662-Data.db
 ERROR 11:05:17,389 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 72392391 
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 72392391 
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
 I deleted the sstable with the broken key (RF==3, so I figured I could just 
 repair when it came back up), but now instead I get:
 ERROR 11:16:02,045 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 02473253(
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at

[jira] Commented: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: 02473253(


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002108#comment-13002108
 ] 

Joaquin Casares commented on CASSANDRA-1170:


ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java (line 
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main] 
java.lang.RuntimeException: java.lang.NumberFormatException: For input 
string: 981834x22 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
 
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
at java.lang.Thread.run(Thread.java:619) 
Caused by: java.lang.NumberFormatException: For input string: 981834x22 
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
at java.lang.Integer.parseInt(Integer.java:458) 
at java.math.BigInteger.init(BigInteger.java:325) 
at java.math.BigInteger.init(BigInteger.java:451) 
at 
org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32) 
at 
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 
at 
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
 
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62) 
at 
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
 
at 
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
 

This was seen on a 0.6.8 cluster which is fairly fresh.

The Hector code that was used to run this was:
private void buildBatchMutation(BatchMutation batch, 
String columnFamily, 
String key, byte [] superColumnKey, byte 
[] columnKey, byte [] columnVal) 
{ 
  ListString columnFamilies = new ArrayListString();

  columnFamilies.add(columnFamily);

  if (columnVal == null) 
  { 
Deletion deletion = new Deletion(); 
SlicePredicate slicePredicate = new SlicePredicate(); 
Listbyte[] columnList = new ArrayListbyte[]();

columnList.add(columnKey); 
slicePredicate.setColumn_names(columnList);

if (superColumnKey != null) 
deletion.setSuper_column(superColumnKey);

deletion.predicate = slicePredicate; 
deletion.setTimestamp(System.currentTimeMillis() * 1000);

batch.addDeletion(key, columnFamilies, deletion); 
  } 
  else 
  { 
Column column = new Column();

column.setName(columnKey); 
column.setValue(columnVal); 
column.setTimestamp(System.currentTimeMillis() * 1000);

if (superColumnKey == null) 
  batch.addInsertion(key, columnFamilies, column); 
else 
{ 
  SuperColumn superColumn = new SuperColumn(); 
  ListColumn columns = new ArrayListColumn();

  columns.add(column); 
  superColumn.setColumns(columns); 
  superColumn.setName(superColumnKey);

  batch.addSuperInsertion(key, columnFamilies, superColumn); 
} 
  } 
}
If this helps any.

 java.lang.NumberFormatException: For input string: 02473253(
 --

 Key: CASSANDRA-1170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6.2, 0.6.8
 Environment: Cassandra 0.6.2
 Ubuntu 9.10 on EC2
 java version 1.6.0_0
 IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
 OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
Reporter: David King

 When trying to boot a node:
  INFO 11:05:16,117 Sampling index for 
 /cassandra/data/permacache/permacache-13662-Data.db
 ERROR 11:05:17,389 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 72392391 
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002126#comment-13002126
 ] 

Sylvain Lebresne commented on CASSANDRA-2231:
-

{quote}
Does this mean that I can have any number of components in my dynamic composite 
key and that b and t are aliases to BytesType and TimeUUIDType for the 
purpose of space efficienct? Using both or neither of those aliases is valid 
and the order in which I use them isn't mandated, correct?
{quote}

That is correct. The aliases are here only for space efficiency. No obligation 
of using them of any sort. 

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: 02473253(


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joaquin Casares updated CASSANDRA-1170:
---

Affects Version/s: (was: 0.6.8)

 java.lang.NumberFormatException: For input string: 02473253(
 --

 Key: CASSANDRA-1170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.6.2
 Environment: Cassandra 0.6.2
 Ubuntu 9.10 on EC2
 java version 1.6.0_0
 IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
 OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
Reporter: David King

 When trying to boot a node:
  INFO 11:05:16,117 Sampling index for 
 /cassandra/data/permacache/permacache-13662-Data.db
 ERROR 11:05:17,389 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 72392391 
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 72392391 
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
 I deleted the sstable with the broken key (RF==3, so I figured I could just 
 repair when it came back up), but now instead I get:
 ERROR 11:16:02,045 Exception encountered during startup.
 java.lang.NumberFormatException: For input string: 02473253(
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:481)
 at java.math.BigInteger.init(BigInteger.java:343)
 at java.math.BigInteger.init(BigInteger.java:467)
 at 
 org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32)
 at 
 org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 at 
 org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
 at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:178)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
 at org.apache.cassandra.db.Table.init(Table.java:338)
 at org.apache.cassandra.db.Table.open(Table.java:199)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
 at

[jira] Issue Comment Edited: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: 02473253(


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002108#comment-13002108
 ] 

Joaquin Casares edited comment on CASSANDRA-1170 at 3/3/11 6:27 PM:


This was seen on a 0.6.8 cluster which is fairly fresh.

ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java (line 
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main] 
java.lang.RuntimeException: java.lang.NumberFormatException: For input 
string: 981834x22 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
 
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
at java.lang.Thread.run(Thread.java:619) 
Caused by: java.lang.NumberFormatException: For input string: 981834x22 
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
at java.lang.Integer.parseInt(Integer.java:458) 
at java.math.BigInteger.init(BigInteger.java:325) 
at java.math.BigInteger.init(BigInteger.java:451) 
at 
org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32) 
at 
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 
at 
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
 
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62) 
at 
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
 
at 
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
 

The Hector code that was used to run this was:
private void buildBatchMutation(BatchMutation batch, 
String columnFamily, 
String key, byte [] superColumnKey, byte 
[] columnKey, byte [] columnVal) 
{ 
  ListString columnFamilies = new ArrayListString();

  columnFamilies.add(columnFamily);

  if (columnVal == null) 
  { 
Deletion deletion = new Deletion(); 
SlicePredicate slicePredicate = new SlicePredicate(); 
Listbyte[] columnList = new ArrayListbyte[]();

columnList.add(columnKey); 
slicePredicate.setColumn_names(columnList);

if (superColumnKey != null) 
deletion.setSuper_column(superColumnKey);

deletion.predicate = slicePredicate; 
deletion.setTimestamp(System.currentTimeMillis() * 1000);

batch.addDeletion(key, columnFamilies, deletion); 
  } 
  else 
  { 
Column column = new Column();

column.setName(columnKey); 
column.setValue(columnVal); 
column.setTimestamp(System.currentTimeMillis() * 1000);

if (superColumnKey == null) 
  batch.addInsertion(key, columnFamilies, column); 
else 
{ 
  SuperColumn superColumn = new SuperColumn(); 
  ListColumn columns = new ArrayListColumn();

  columns.add(column); 
  superColumn.setColumns(columns); 
  superColumn.setName(superColumnKey);

  batch.addSuperInsertion(key, columnFamilies, superColumn); 
} 
  } 
}
If this helps any. Properly formatted: http://pastebin.com/SGu9AuDV

  was (Author: j.casares):
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java 
(line 
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main] 
java.lang.RuntimeException: java.lang.NumberFormatException: For input 
string: 981834x22 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
 
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
at java.lang.Thread.run(Thread.java:619) 
Caused by: java.lang.NumberFormatException: For input string: 981834x22 
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
at java.lang.Integer.parseInt(Integer.java:458) 
at java.math.BigInteger.init(BigInteger.java:325) 
at java.math.BigInteger.init(BigInteger.java:451) 
at 
org.apache.cassandra.dht.BigIntegerToken.init(BigIntegerToken.java:32) 
at 
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
 
at 
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
 
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62) 
at 
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
 
at 
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
 
at 
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
 

This was seen on a 0.6.8 cluster which is

[jira] Created: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08

2011-03-03 Thread Pavel Yaskevich (JIRA)

Fix distributed tests after updating Guava to 08


 Key: CASSANDRA-2265
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich


Distributed tests fail to start cluster:
{code}
[junit] java.lang.NoSuchMethodError: 
com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
[junit] at 
org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
[junit] at 
org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
[junit] at 
org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
[junit] at 
org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
[junit] at 
org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
[junit] at 
org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
[junit] at $Proxy58.objectExists(Unknown Source)
[junit] at 
org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
[junit] at $Proxy59.objectExists(Unknown Source)
[junit] at 
org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
[junit] at 
org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
[junit] at 
org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
[junit] at 
org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
[junit] at 
org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
[junit] at 
org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
[junit] at 
org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
[junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
[junit] at 
org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
[junit] at $Proxy59.objectExists(Unknown Source)
[junit] at 
org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
[junit] at 
org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
[junit] at 
org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
[junit] at 
org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
[junit] at 
org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
[junit] at 
org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
[junit] at 
org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
[junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
{code}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076729 [2/2] - in /cassandra/trunk: ./ contrib/ debian/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/columnit

Modified: 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java?rev=1076729r1=1076728r2=1076729view=diff
==
--- 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java 
(original)
+++ 
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java 
Thu Mar  3 19:11:17 2011
@@ -307,7 +307,7 @@ public class ColumnFamilyStoreTest exten
 }
 
 @Test
-public void testIndexCreate() throws IOException, ConfigurationException, 
InterruptedException
+public void testIndexCreate() throws IOException, ConfigurationException, 
InterruptedException, ExecutionException
 {
 Table table = Table.open(Keyspace1);
 
@@ -324,6 +324,9 @@ public class ColumnFamilyStoreTest exten
 while (!SystemTable.isIndexBuilt(Keyspace1, 
cfs.getIndexedColumnFamilyStore(ByteBufferUtil.bytes(birthdate)).columnFamily))
 TimeUnit.MILLISECONDS.sleep(100);
 
+// we had a bug (CASSANDRA-2244) where index would get created but not 
flushed -- check for that  
+assert cfs.getIndexedColumnFamilyStore(cd.name).getSSTables().size()  
0;
+
 IndexExpression expr = new 
IndexExpression(ByteBufferUtil.bytes(birthdate), IndexOperator.EQ, 
ByteBufferUtil.bytes(1L));
 IndexClause clause = new IndexClause(Arrays.asList(expr), 
ByteBufferUtil.EMPTY_BYTE_BUFFER, 100);
 IFilter filter = new IdentityQueryFilter();
@@ -331,8 +334,7 @@ public class ColumnFamilyStoreTest exten
 Range range = new Range(p.getMinimumToken(), p.getMinimumToken());
 ListRow rows = table.getColumnFamilyStore(Indexed2).scan(clause, 
range, filter);
 assert rows.size() == 1 : StringUtils.join(rows, ,);
-String key = new 
String(rows.get(0).key.key.array(),rows.get(0).key.key.position(),rows.get(0).key.key.remaining());
 
-assert k1.equals( key );
+assertEquals(k1, ByteBufferUtil.string(rows.get(0).key.key));
 }
 
 @Test

Modified: cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java?rev=1076729r1=1076728r2=1076729view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java 
(original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java Thu Mar 
 3 19:11:17 2011
@@ -1,4 +1,25 @@
 package org.apache.cassandra.db;
+/*
+ * 
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * 
+ */
+
 
 import java.io.IOException;
 import java.util.concurrent.ExecutionException;

Modified: cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java?rev=1076729r1=1076728r2=1076729view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java (original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java Thu Mar  3 
19:11:17 2011
@@ -1,4 +1,25 @@
 package org.apache.cassandra.db;
+/*
+ * 
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * 
+ */
+
 
 import

[jira] Updated: (CASSANDRA-1034) Remove assumption that Key to Token is one-to-one

[
https://issues.apache.org/jira/browse/CASSANDRA-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sylvain Lebresne updated CASSANDRA-1034:

Attachment: 0002-LengthPartitioner.patch
0001-Make-range-accept-both-Token-and-DecoratedKey.patch

Attaching a patch that I believe solves this. It makes Range accept both Token
and DecoratedKey and makes those two compare together correctly.

It introduces a new marker interface (RingPosition) instead of making
DecoratedKey extends Token (for reason explained in the comments of
RingPosition but to sum up: I think it's cleaner).

The second patch attached is just a stupid partitioner that use for token the
length of the key. It's just for testing and not meant for inclusion. But this
shows that with the first patch, you can do correct range query that go from
'the middle of a token' to the 'middle of another one'.

An important note is that this breaks the serialization unit tests, because now
an AbstractBounds can use decoratedKeys, and thus serialized AbstractBounds are
incompatible with previous version. Not sure how to deal with that though, I
though we had a plan for dealing with that but I'll admit I don't remember the
details.

Remove assumption that Key to Token is one-to-one
-

Key: CASSANDRA-1034
URL: https://issues.apache.org/jira/browse/CASSANDRA-1034
Project: Cassandra
Issue Type: Bug
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
Attachments:
0001-Make-range-accept-both-Token-and-DecoratedKey.patch,
0002-LengthPartitioner.patch, 1034_v1.txt

get_range_slices assumes that Tokens do not collide and converts a KeyRange
to an AbstractBounds. For RandomPartitioner, this assumption isn't safe, and
would lead to a very weird heisenberg.
Converting AbstractBounds to use a DecoratedKey would solve this, because the
byte[] key portion of the DecoratedKey can act as a tiebreaker.
Alternatively, we could make DecoratedKey extend Token, and then use
DecoratedKeys in places where collisions are unacceptable.

--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal

2011-03-03 Thread Ed Anuff (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002144#comment-13002144
 ] 

Ed Anuff commented on CASSANDRA-2231:
-

bq.Greater-than is already doable in my previous patch (up to the bug in 
validation). For the less-than part, I agree that it is nice to be able to do 
it easily. In my new patch, I add a leading byte to each component, whose 
purpose is to always be 0, except for lesser-than query. That way, you can do 
the query above easily. The price is a slightly more complicated encoding but I 
think it's totally worth it.

Just to be clear, the original idea was to make it possible to construct a key 
for the purposes of doing a range slice that would compare inclusive either or 
both at the start and finish of the range.  This appears to be possible with 
the inclusion byte that you're using in lines 179 through 184 of your patch.  
Is that correct?

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002147#comment-13002147
 ] 

Sylvain Lebresne commented on CASSANDRA-2231:
-

Yes that is also correct.

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1684) Entity groups


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002155#comment-13002155
 ] 

Sylvain Lebresne commented on CASSANDRA-1684:
-

bq. Do tokens have to be one-to-one unique with keys, or could you have 
multiple keys share the same token? (apparently that's currently possible, 
although an extreme edge case, with the RandomPartitioner)

Right now, they do have to be one-to-one. That's the 'raison d'être' of 
CASSANDRA-1034 (and I won't hide that my interest for the latter is motivated 
by this ticket, even though we should fix it because of RandomPartioner anyway).

As for this ticket, I think using parts of the key for the token is only the 
first step (but an important one). The main thing we want here is to apply 
mutation on an entity group consistently, that is in one commit log 
transaction. That in turn is not very complicated in theory, but will be much 
more work in practice I believe.

As a side note, I think it would also be nice to find a trick to make this 
work with the existing partitioners. Otherwise, since we can't change 
partitioners, this would make this useful for only new clusters, which would be 
sad.


 Entity groups
 -

 Key: CASSANDRA-1684
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1684
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 0.8

   Original Estimate: 80h
  Remaining Estimate: 80h

 Supporting entity groups similar to App Engine's (that is, allow rows to be 
 part of a parent entity group, whose key is used for routing instead of the 
 row itself) allows several improvements:
  - batches within an EG can be atomic across multiple rows
  - order-by-value queries within an EG only have to touch a single replica 
 even with RandomPartitioner

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-1828) Create a pig storefunc


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-1828:


Attachment: (was: 0001-add-storage-ability-to-pig-CassandraStorage.txt)

 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0002-Fix-build-bin-script.txt, 
 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Issue Comment Edited: (CASSANDRA-1828) Create a pig storefunc


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002162#comment-13002162
 ] 

Brandon Williams edited comment on CASSANDRA-1828 at 3/3/11 7:35 PM:
-

Couldn't get Eldon's patch to apply, but updated 0001 with his changes to add 
deletions and explicitly cast String, as well as other cleanups.  Only 0001 and 
0002 are part of the patchset, 0003 is an outdated conglomeration of the two 
now.

  was (Author: brandon.williams):
Couldn't get Eldon's patch to apply, but update 0001 with his changes to 
add deletions and explicitly cast String, as well as other cleanups.  Only 0001 
and 0002 are part of the patchset, 0003 is an outdated conglomeration of the 
two now.
  
 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-1828) Create a pig storefunc


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-1828:


Attachment: 0001-add-storage-ability-to-pig-CassandraStorage.txt

Couldn't get Eldon's patch to apply, but update 0001 with his changes to add 
deletions and explicitly cast String, as well as other cleanups.  Only 0001 and 
0002 are part of the patchset, 0003 is an outdated conglomeration of the two 
now.

 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal

2011-03-03 Thread Ed Anuff (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002170#comment-13002170
 ] 

Ed Anuff commented on CASSANDRA-2231:
-

If two dynamic composite types are compared, the first integer,uuid,timestamp 
and the second string,uuid,timestamp, this results in an exception being 
thrown in line 659, correct?  In the original CompositeType, the component 
types each had an ordinal type value and the comparison was done on those type 
values if the components were of different types.  I might suggest that in your 
code using the alias character byte or the hashCode() of the classname as the 
type value and doing a similar comparison, rather than throwing an exception.

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08

2011-03-03 Thread Pavel Yaskevich (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002174#comment-13002174
 ] 

Pavel Yaskevich commented on CASSANDRA-2265:


Created an issue in jclouds tracker 
http://code.google.com/p/jclouds/issues/detail?id=492

 Fix distributed tests after updating Guava to 08
 

 Key: CASSANDRA-2265
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 0.8


 Distributed tests fail to start cluster:
 {code}
 [junit] java.lang.NoSuchMethodError: 
 com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
 [junit]   at 
 org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
 [junit]   at 
 org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
 [junit]   at $Proxy58.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 {code}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08

2011-03-03 Thread Pavel Yaskevich (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-2265:
---

Priority: Blocker  (was: Major)

 Fix distributed tests after updating Guava to 08
 

 Key: CASSANDRA-2265
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Priority: Blocker
 Fix For: 0.8


 Distributed tests fail to start cluster:
 {code}
 [junit] java.lang.NoSuchMethodError: 
 com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
 [junit]   at 
 org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
 [junit]   at 
 org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
 [junit]   at $Proxy58.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 {code}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002193#comment-13002193
 ] 

Sylvain Lebresne commented on CASSANDRA-2231:
-

We could do that. But I have to admit that I'm usually not very fond of making 
things work when it's not clear they should. I think that could hide problems 
more than it helps. Better fail fast if there is a mess up.

And if you want to store in the same row column names that have intrinsically 
different structures, you can prefix it manually to distinguish thinks. That 
is, in you example, you'll replace integer,uuid,timestamp (type1) by 
integer,integer,uuid,timestamp and string,uuid,timestamp (type2) by 
integer,string,uuid,timestamp and assign the same integer for all the column 
name of type1 and the same integer (but a different from first one) to all 
those of type2. And this has actually a bunch of advantages: because you use a 
true component, you can query for the whole range of column name having the 
same 'type' (aka all type1 or all type2). Moreover you do control the prefix so 
you have more control on how each separate set of columns sort between each 
other. Yes you have to manage those prefix client side, but honestly I think 
it's a small price for the type safety it provides.


 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002196#comment-13002196
 ] 

Gary Dusbabek commented on CASSANDRA-2262:
--

bq. Last remark, the tests of the first patch are testing the wrong equation.
Can you elaborate?

as for testInteger2(), that should produce the buffer error as you later 
expected.

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java

Add standard deviation to column names/values in stress.java


 Key: CASSANDRA-2266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2266
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Reporter: Brandon Williams
Assignee: Pavel Yaskevich
Priority: Minor


stress currently produce fixed-size columns and values across rows, leading to 
unnatural problems such as the memtables turning over at the same time on all 
nodes.  Instead these could take standard deviation flags.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2267) stress.java daemon mode

stress.java daemon mode
---

 Key: CASSANDRA-2267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2267
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Pavel Yaskevich


stress.java uses a JVM, but there is no way to warm it up, which skews results. 
 It would be nice if there was some sort of daemon mode so the JVM could stay 
hot, and even better if there was a way to control the daemon programmatically, 
perhaps via JMX.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002197#comment-13002197
 ] 

Gary Dusbabek commented on CASSANDRA-2228:
--

aha. I was looking in the trunk code (no chance of null).  I see the problem in 
the 0.7 branch.

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2) If I restart then, I'm getting an Nullpointer exception. The OOM error 
 will only appear once.
 ERROR [main]

[jira] Commented: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002199#comment-13002199
 ] 

Jonathan Ellis commented on CASSANDRA-2266:
---

ISTM that we could get 99% of the benefit here w/o too much complexity by 
decreeing that column value sizes go from 0..2*size, i.e., average stays the 
same but you do get all kinds of sizes in there.  I.e., I don't think I'm in 
favor of adding more options.

 Add standard deviation to column names/values in stress.java
 

 Key: CASSANDRA-2266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2266
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Reporter: Brandon Williams
Assignee: Pavel Yaskevich
Priority: Minor

 stress currently produce fixed-size columns and values across rows, leading 
 to unnatural problems such as the memtables turning over at the same time on 
 all nodes.  Instead these could take standard deviation flags.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002203#comment-13002203
 ] 

Brandon Williams commented on CASSANDRA-2266:
-

That sounds reasonable to me.

 Add standard deviation to column names/values in stress.java
 

 Key: CASSANDRA-2266
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2266
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Reporter: Brandon Williams
Assignee: Pavel Yaskevich
Priority: Minor

 stress currently produce fixed-size columns and values across rows, leading 
 to unnatural problems such as the memtables turning over at the same time on 
 all nodes.  Instead these could take standard deviation flags.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2267) stress.java daemon mode

2011-03-03 Thread Eric Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002202#comment-13002202
 ] 

Eric Evans commented on CASSANDRA-2267:
---

bq. ...a way to control the daemon programmatically, perhaps via JMX.

I love JMX, but everytime you use it a Ruby dev kills a kitten.

 stress.java daemon mode
 ---

 Key: CASSANDRA-2267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2267
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Pavel Yaskevich

 stress.java uses a JVM, but there is no way to warm it up, which skews 
 results.  It would be nice if there was some sort of daemon mode so the JVM 
 could stay hot, and even better if there was a way to control the daemon 
 programmatically, perhaps via JMX.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2268) CQL-enabled stress.java

2011-03-03 Thread Eric Evans (JIRA)

CQL-enabled stress.java
---

 Key: CASSANDRA-2268
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2268
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Eric Evans
Priority: Minor
 Fix For: 0.8


It would be great if stress.java had a CQL mode.  For making the inevitable 
RPC-CQL comparisons, but also as a basis for measuring optimizations, and 
spotting performance regressions.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2267) stress.java daemon mode


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002206#comment-13002206
 ] 

Brandon Williams commented on CASSANDRA-2267:
-

It doesn't *have* to be JMX, but that seems like the easiest path initially.

 stress.java daemon mode
 ---

 Key: CASSANDRA-2267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2267
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Pavel Yaskevich

 stress.java uses a JVM, but there is no way to warm it up, which skews 
 results.  It would be nice if there was some sort of daemon mode so the JVM 
 could stay hot, and even better if there was a way to control the daemon 
 programmatically, perhaps via JMX.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2267) stress.java daemon mode


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002204#comment-13002204
 ] 

Brandon Williams commented on CASSANDRA-2267:
-

To clarify, this isn't so much about warming the JVM, but that is a side effect 
gained from having a daemon mode and the operations exposed via an API: the 
operator can warm it sufficiently themselves.

 stress.java daemon mode
 ---

 Key: CASSANDRA-2267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2267
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Pavel Yaskevich

 stress.java uses a JVM, but there is no way to warm it up, which skews 
 results.  It would be nice if there was some sort of daemon mode so the JVM 
 could stay hot, and even better if there was a way to control the daemon 
 programmatically, perhaps via JMX.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2268) CQL-enabled stress.java


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-2268:
-

Assignee: Pavel Yaskevich

 CQL-enabled stress.java
 ---

 Key: CASSANDRA-2268
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2268
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Eric Evans
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: cql
 Fix For: 0.8


 It would be great if stress.java had a CQL mode.  For making the inevitable 
 RPC-CQL comparisons, but also as a basis for measuring optimizations, and 
 spotting performance regressions.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal

2011-03-03 Thread Ed Anuff (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002226#comment-13002226
 ] 

Ed Anuff commented on CASSANDRA-2231:
-

Hmm, that would work, although you certainly wouldn't want to use the LongType 
as your integer.  I guess the minimum overhead for a component is 6 bytes - 2 
header, 2 length, 1 value, 1 inclusion flag.

I'm not seeing anything else that wouldn't let me use this as a functional 
replacement for the original CompositeType, so I'm +1 on it.  Thanks!

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2248) ant javadoc fails on windows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2248:
-

Attachment: (was: v1-0001-initialize-localEndpoint_.txt)

 ant javadoc fails on windows
 

 Key: CASSANDRA-2248
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2248
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
 Environment: windows 7, ant 1.8.2
Reporter: Norman Maurer
Assignee: Norman Maurer
 Fix For: 0.7.3

 Attachments: CASSANDRA-2248.diff


 When try to run ant javadoc (or any task that include javadoc) on windows 
 it fails with the error:
 Javadoc failed: java.io.IOException: Cannot run program c:\Program 
 Files\Java\jdk1.6.0_17\bin\javadoc.exe: CreateProcess error=87, The 
 parameter is incorrect

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2248) ant javadoc fails on windows


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2248:
-

Attachment: v1-0001-initialize-localEndpoint_.txt

 ant javadoc fails on windows
 

 Key: CASSANDRA-2248
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2248
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
 Environment: windows 7, ant 1.8.2
Reporter: Norman Maurer
Assignee: Norman Maurer
 Fix For: 0.7.3

 Attachments: CASSANDRA-2248.diff


 When try to run ant javadoc (or any task that include javadoc) on windows 
 it fails with the error:
 Javadoc failed: java.io.IOException: Cannot run program c:\Program 
 Files\Java\jdk1.6.0_17\bin\javadoc.exe: CreateProcess error=87, The 
 parameter is incorrect

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2228:
-

Attachment: v1-0001-initialize-localEndpoint_.txt

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4

 Attachments: v1-0001-initialize-localEndpoint_.txt


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2) If I restart then, I'm getting an Nullpointer exception. The OOM error 
 will only appear once.
 ERROR [main] 2011-02-23 16:42:32,782

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002253#comment-13002253
 ] 

Gary Dusbabek commented on CASSANDRA-2124:
--

bq. For this, either we need to introduce a wrapper or Ascii, BytesType, UUID 
etc. classes need to provide a way for toString() calls.

for AsciiType use getString(), for BytesType use getBytes(), for UUID use 
getObject and cast to a java.util.UUID.

I think this patch set is ready to commit and we can move on to the next 
iteration (ResultSetMetadata or whatever).

 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v1-0001-first-pass-at-column-decoding.txt, 
 v1-0002-clean-up-JdbcDriverTest.txt, 
 v1-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v1-0004-more-comments-in-ColumnDecoder.txt


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002254#comment-13002254
 ] 

Sylvain Lebresne commented on CASSANDRA-2231:
-

Yes, IntegerType would be a better choice because it uses BigInteger that packs 
stuffs as much as it can (aka, 1 byte if it fits). I know that because it 
bitted me while writing the unit test for this :))

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Priority: Minor
 Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, 
 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002258#comment-13002258
 ] 

Sylvain Lebresne commented on CASSANDRA-2262:
-

bq. Can you elaborate?

I mean that the ticket description said that we should have get(from(s)) == s, 
but some of the test in RoundTripTest actually test from(get(b)) == b (all of 
them except for the two UUID tests). So this is not really testing what it 
should. And this is not the same since for instance, get(from(s)) always work 
for IntegerType while from(get(b)) can throw an exception (and thus not work) 
depending on what is b as shown by testInteger2.

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002268#comment-13002268
 ] 

Jonathan Ellis commented on CASSANDRA-2228:
---

+1

 Race conditions when reinitialisating nodes (OOM + Nullpointer)
 ---

 Key: CASSANDRA-2228
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
 Fix For: 0.7.4

 Attachments: v1-0001-initialize-localEndpoint_.txt


 I had a corrupt system table which wouldn't compact anymore and I deleted the 
 files and restarted cassandra and let it take the same token/ip address.
 I experienced the same errors when I'm adding a newly installed node under 
 the same token/ip address before calling repair.
 1)
 After a few seconds/minutes, I get a OOM error:
  INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164) 
 Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133) 
 Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace: 
 table_userentriesrep factor:3rep 
 strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
  
 org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
  INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Migrations at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1 
 operations)
  INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) 
 Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line 
 666) switching in a fresh Memtable for Schema at 
 CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log', 
 position=226075)
  INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line 
 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
  INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89) 
 table_sourcedetection.table_sourcedetection 0,0   
   0/00/20
 ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java 
 (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
 java.lang.OutOfMemoryError: Java heap space
 at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
 at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
 at 
 org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
 at 
 org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
 at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
 at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 2) If I restart then, I'm getting an Nullpointer exception. The OOM error 
 will only appear once.
 ERROR [main] 2011-02-23 16:42:32,782

[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002271#comment-13002271
 ] 

Jonathan Ellis commented on CASSANDRA-2262:
---

bq. the ticket description said that we should have get(from(s)) == s, but some 
of the test in RoundTripTest actually test from(get(b)) == b

don't we want to test both?

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002275#comment-13002275
 ] 

Gary Dusbabek commented on CASSANDRA-2262:
--

Yes. I'm reworking those tests to do just that.

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
v2-0003-compose-method-for-AbstractTypes.txt
v2-0002-BytesType.fromString-expects-a-hex-string.txt
v2-0001-test-shows-no-roundtrip-in-BytesType.txt

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v1-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v1-0003-compose-method-for-AbstractTypes.txt, 
 v1-0004-assume-utf8-in-CliTest-keys-dammit.txt, 
 v2-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v2-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v2-0003-compose-method-for-AbstractTypes.txt, 
 v2-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: (was: v1-0003-compose-method-for-AbstractTypes.txt)

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v2-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v2-0003-compose-method-for-AbstractTypes.txt, 
 v2-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: (was: v1-0001-test-shows-no-roundtrip-in-BytesType.txt)

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v2-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v2-0003-compose-method-for-AbstractTypes.txt, 
 v2-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: (was: v1-0004-assume-utf8-in-CliTest-keys-dammit.txt)

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v2-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v2-0003-compose-method-for-AbstractTypes.txt, 
 v2-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2262:
-

Attachment: (was: v1-0002-BytesType.fromString-expects-a-hex-string.txt)

 use o.a.c.marshal.*Type for CQL 
 

 Key: CASSANDRA-2262
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.8

 Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt, 
 v2-0002-BytesType.fromString-expects-a-hex-string.txt, 
 v2-0003-compose-method-for-AbstractTypes.txt, 
 v2-0004-assume-utf8-in-CliTest-keys-dammit.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a 
 parsed term to a {{ByteBuffer}} of the right contents, this should be moved 
 into the individual {{AbstractType}} subclasses (aka 
 {{AbstractType.fromString}}).
 Additionally, a method that does the inverse (returning the string 
 equivalent), needs to exists such that 
 {{AbstractType.getString(AbstractType.fromString(s)) == s}}
 Finally, for use in results decoding a method should exist that given a 
 {{ByteBuffer}} will return the appropriate object for that type.  For 
 example, {{IntegerType.compose(ByteBuffer) - java.math.BigInteger}}, or 
 {{LexicalUUIDType.compose(ByteBuffer) - java.util.UUID}}.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076866 - in /cassandra/branches/cassandra-0.7: ./ src/java/org/apache/cassandra/gms/ src/java/org/apache/cassandra/service/ test/unit/org/apache/cassandra/cli/ test/unit/org/apache/cassa

2011-03-03 Thread gdusbabek

Author: gdusbabek
Date: Thu Mar  3 22:54:56 2011
New Revision: 1076866

URL: http://svn.apache.org/viewvc?rev=1076866view=rev
Log:
initialize localendpoint in gossiper earlier. patch by gdusbabek, reviewed by 
jbellis. CASSANDRA-2228

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/service/RemoveTest.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076866r1=1076865r2=1076866view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Mar  3 22:54:56 2011
@@ -1,7 +1,7 @@
 0.7.4
  * add nodetool join command (CASSANDRA-2160)
  * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244)
-
+ * initialize endpoing in gossiper earlier (CASSANDRA-2228)
 
 0.7.3
  * Keep endpoint state until aVeryLongTime (CASSANDRA-2115)

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java?rev=1076866r1=1076865r2=1076866view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
 Thu Mar  3 22:54:56 2011
@@ -26,6 +26,7 @@ import java.util.*;
 import java.util.Map.Entry;
 import java.util.concurrent.*;
 
+import org.apache.cassandra.utils.FBUtilities;
 import org.cliffc.high_scale_lib.NonBlockingHashMap;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -149,6 +150,7 @@ public class Gossiper implements IFailur
 
 private Gossiper()
 {
+localEndpoint_ = FBUtilities.getLocalAddress();
 // 3 days
 aVeryLongTime_ = 259200 * 1000;
 // half of QUARATINE_DELAY, to ensure justRemovedEndpoints has enough 
leeway to prevent re-gossip
@@ -870,14 +872,13 @@ public class Gossiper implements IFailur
  * Start the gossiper with the generation # retrieved from the System
  * table
  */
-public void start(InetAddress localEndpoint, int generationNbr)
+public void start(int generationNbr)
 {
-localEndpoint_ = localEndpoint;
 /* Get the seeds from the config and initialize them. */
 SetInetAddress seedHosts = DatabaseDescriptor.getSeeds();
 for (InetAddress seed : seedHosts)
 {
-if (seed.equals(localEndpoint))
+if (seed.equals(localEndpoint_))
 continue;
 seeds_.add(seed);
 }

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java?rev=1076866r1=1076865r2=1076866view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
 Thu Mar  3 22:54:56 2011
@@ -282,7 +282,7 @@ public class StorageService implements I
 if (!initialized)
 {
 logger_.warn(Starting gossip by operator request);
-Gossiper.instance.start(FBUtilities.getLocalAddress(), 
(int)(System.currentTimeMillis() / 1000));
+Gossiper.instance.start((int)(System.currentTimeMillis() / 1000));
 initialized = true;
 }
 }
@@ -343,7 +343,7 @@ public class StorageService implements I
 logger_.info(Starting up client gossip);
 setMode(Client, false);
 Gossiper.instance.register(this);
-Gossiper.instance.start(FBUtilities.getLocalAddress(), 
(int)(System.currentTimeMillis() / 1000)); // needed for node-ring gathering.
+Gossiper.instance.start((int)(System.currentTimeMillis() / 1000)); // 
needed for node-ring gathering.
 MessagingService.instance().listen(FBUtilities.getLocalAddress());
 
 // sleep a while to allow gossip to warm up (the other nodes need to 
know about this one before they can reply).
@@ -418,7 +418,7 @@ public class StorageService implements I
 // (we won't be part of the storage ring though until we add a nodeId 
to our state, below.)
 Gossiper.instance.register(this);

svn commit: r1076868 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/gms/ test/unit/org/apache/cassandra/cli/

2011-03-03 Thread gdusbabek

Author: gdusbabek
Date: Thu Mar  3 22:59:36 2011
New Revision: 1076868

URL: http://svn.apache.org/viewvc?rev=1076868view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java
cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Mar  3 22:59:36 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1071777
-/cassandra/branches/cassandra-0.7:1026516-1076706
+/cassandra/branches/cassandra-0.7:1026516-1076866
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1076868r1=1076867r2=1076868view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Thu Mar  3 22:59:36 2011
@@ -14,7 +14,7 @@
 0.7.4
  * add nodetool join command (CASSANDRA-2160)
  * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244)
-
+ * initialize endpoing in gossiper earlier (CASSANDRA-2228)
 
 0.7.3
  * Keep endpoint state until aVeryLongTime (CASSANDRA-2115)

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Mar  3 22:59:36 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1076706
+/cassandra/branches/cassandra-0.7/contrib:1026516-1076866
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Mar  3 22:59:36 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1071777
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1076706
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1076866
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Mar  3 22:59:36 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1071777
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1076706
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1076866
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
---

[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002303#comment-13002303
 ] 

Jonathan Ellis commented on CASSANDRA-1969:
---

Interesting, it's exactly 2x the 64M you pasted: default 
sun.misc.VM.maxDirectMemory()) == 129957888.

So allocating 90MB works fine but 256M fails with OOM.

 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-1828) Create a pig storefunc

2011-03-03 Thread Jeremy Hanna (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-1828:


Reviewer: jeromatron

 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1828) Create a pig storefunc

2011-03-03 Thread Jeremy Hanna (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002328#comment-13002328
 ] 

Jeremy Hanna commented on CASSANDRA-1828:
-

+1

2 comments:

- I really like exceptions for invalid configuration - e.g. 
PIG_INITIAL_ADDRESS environment variable not set
- why not just have getOutputFormat just return new ColumnFamilyOutputFormat();

 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-03-03 Thread Vijay (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002330#comment-13002330
 ] 

Vijay commented on CASSANDRA-1969:
--

For single allocation thats true the following code works!... but wont our 
byte[] within a column less than that? is this a real concern?

final int SIZE = 1024;
final int KBS = 1024 * 1024;

for (int i = 0; i  KBS; i++) {
  ByteBuffer.allocateDirect(SIZE );
  System.out.println(allocating: + i);
}

System.out.println(ok);
Thread.sleep(1);
  }





 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server

OOM in the Thrift thread doesn't shut down server
-

 Key: CASSANDRA-2269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.4


Example:

{noformat}
java.lang.OutOfMemoryError: Java heap space
at 
org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
at 
org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
at 
org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
at 
org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
at 
org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
at 
org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
at 
org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
at 
org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
at 
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
at 
org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
at 
org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
at 
org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
at 
org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
at 
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2269:
--

Fix Version/s: 0.6.13

 OOM in the Thrift thread doesn't shut down server
 -

 Key: CASSANDRA-2269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.6.13, 0.7.4


 Example:
 {noformat}
 java.lang.OutOfMemoryError: Java heap space
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
 at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
 at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
 at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
 at 
 org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
 at 
 org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
 at 
 org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
 at 
 org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
 at 
 org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
 at 
 org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
 at 
 org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
 at 
 org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
 at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
 Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
 {noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2170) Load spikes


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2170:
--

Fix Version/s: (was: 0.6.12)
   0.6.13

 Load spikes
 ---

 Key: CASSANDRA-2170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2170
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.6.11
Reporter: Jonathan Ellis
 Fix For: 0.6.13


 as reported on CASSANDRA-2058, some users are still seeing load spikes on 
 0.6.11, even with fairly low-volume read workloads.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076891 - in /cassandra/branches/cassandra-0.6: interface/cassandra.thrift src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java

Author: jbellis
Date: Thu Mar  3 23:49:44 2011
New Revision: 1076891

URL: http://svn.apache.org/viewvc?rev=1076891view=rev
Log:
reformat

Modified:
cassandra/branches/cassandra-0.6/interface/cassandra.thrift

cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java

Modified: cassandra/branches/cassandra-0.6/interface/cassandra.thrift
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/interface/cassandra.thrift?rev=1076891r1=1076890r2=1076891view=diff
==
--- cassandra/branches/cassandra-0.6/interface/cassandra.thrift (original)
+++ cassandra/branches/cassandra-0.6/interface/cassandra.thrift Thu Mar  3 
23:49:44 2011
@@ -300,7 +300,7 @@ service Cassandra {
   ColumnOrSuperColumn get(1:required string keyspace,
   2:required string key,
   3:required ColumnPath column_path,
-  4:required ConsistencyLevel consistency_level=ONE)
+  4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
   throws (1:InvalidRequestException ire, 
2:NotFoundException nfe, 3:UnavailableException ue, 4:TimedOutException te),
 
   /**
@@ -311,7 +311,7 @@ service Cassandra {
   2:required string key, 
   3:required ColumnParent column_parent, 
   4:required SlicePredicate predicate, 
-  5:required ConsistencyLevel 
consistency_level=ONE)
+  5:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
 throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -323,7 +323,7 @@ service Cassandra {
   mapstring,ColumnOrSuperColumn multiget(1:required string keyspace, 
2:required liststring keys, 
3:required ColumnPath column_path, 
-   4:required ConsistencyLevel 
consistency_level=ONE)
+   4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
   throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -333,7 +333,7 @@ service Cassandra {
2:required liststring 
keys, 
3:required ColumnParent 
column_parent, 
4:required 
SlicePredicate predicate, 
-   5:required 
ConsistencyLevel consistency_level=ONE)
+   5:required 
ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
 throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -342,7 +342,7 @@ service Cassandra {
   i32 get_count(1:required string keyspace, 
 2:required string key, 
 3:required ColumnParent column_parent, 
-4:required ConsistencyLevel consistency_level=ONE)
+4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
   throws (1:InvalidRequestException ire, 2:UnavailableException ue, 
3:TimedOutException te),
 
   /**
@@ -355,7 +355,7 @@ service Cassandra {
  4:required string start_key=, 
  5:required string finish_key=, 
  6:required i32 row_count=100, 
- 7:required ConsistencyLevel 
consistency_level=ONE)
+ 7:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
  throws (1:InvalidRequestException ire, 2:UnavailableException 
ue, 3:TimedOutException te),
 
   /**
@@ -365,7 +365,7 @@ service Cassandra {
   2:required ColumnParent column_parent, 
   3:required SlicePredicate predicate,
   4:required KeyRange range,
-  5:required ConsistencyLevel 
consistency_level=ONE)
+  5:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
  throws (1:InvalidRequestException ire, 2:UnavailableException 
ue, 3:TimedOutException te),
 
   # modification methods
@@ -380,7 +380,7 @@ service Cassandra {
   3:required ColumnPath column_path, 
   4:required binary value, 
   5:required i64 timestamp, 
-  6:required ConsistencyLevel consistency_level=ONE)
+

svn commit: r1076892 - /cassandra/branches/cassandra-0.6/interface/cassandra.thrift

Author: jbellis
Date: Thu Mar  3 23:50:27 2011
New Revision: 1076892

URL: http://svn.apache.org/viewvc?rev=1076892view=rev
Log:
revert changes to cassandra.thrift for newer compiler version

Modified:
cassandra/branches/cassandra-0.6/interface/cassandra.thrift

Modified: cassandra/branches/cassandra-0.6/interface/cassandra.thrift
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/interface/cassandra.thrift?rev=1076892r1=1076891r2=1076892view=diff
==
--- cassandra/branches/cassandra-0.6/interface/cassandra.thrift (original)
+++ cassandra/branches/cassandra-0.6/interface/cassandra.thrift Thu Mar  3 
23:50:27 2011
@@ -300,7 +300,7 @@ service Cassandra {
   ColumnOrSuperColumn get(1:required string keyspace,
   2:required string key,
   3:required ColumnPath column_path,
-  4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+  4:required ConsistencyLevel consistency_level=ONE)
   throws (1:InvalidRequestException ire, 
2:NotFoundException nfe, 3:UnavailableException ue, 4:TimedOutException te),
 
   /**
@@ -311,7 +311,7 @@ service Cassandra {
   2:required string key, 
   3:required ColumnParent column_parent, 
   4:required SlicePredicate predicate, 
-  5:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+  5:required ConsistencyLevel 
consistency_level=ONE)
 throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -323,7 +323,7 @@ service Cassandra {
   mapstring,ColumnOrSuperColumn multiget(1:required string keyspace, 
2:required liststring keys, 
3:required ColumnPath column_path, 
-   4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+   4:required ConsistencyLevel 
consistency_level=ONE)
   throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -333,7 +333,7 @@ service Cassandra {
2:required liststring 
keys, 
3:required ColumnParent 
column_parent, 
4:required 
SlicePredicate predicate, 
-   5:required 
ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
+   5:required 
ConsistencyLevel consistency_level=ONE)
 throws (1:InvalidRequestException ire, 
2:UnavailableException ue, 3:TimedOutException te),
 
   /**
@@ -342,7 +342,7 @@ service Cassandra {
   i32 get_count(1:required string keyspace, 
 2:required string key, 
 3:required ColumnParent column_parent, 
-4:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+4:required ConsistencyLevel consistency_level=ONE)
   throws (1:InvalidRequestException ire, 2:UnavailableException ue, 
3:TimedOutException te),
 
   /**
@@ -355,7 +355,7 @@ service Cassandra {
  4:required string start_key=, 
  5:required string finish_key=, 
  6:required i32 row_count=100, 
- 7:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+ 7:required ConsistencyLevel 
consistency_level=ONE)
  throws (1:InvalidRequestException ire, 2:UnavailableException 
ue, 3:TimedOutException te),
 
   /**
@@ -365,7 +365,7 @@ service Cassandra {
   2:required ColumnParent column_parent, 
   3:required SlicePredicate predicate,
   4:required KeyRange range,
-  5:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+  5:required ConsistencyLevel 
consistency_level=ONE)
  throws (1:InvalidRequestException ire, 2:UnavailableException 
ue, 3:TimedOutException te),
 
   # modification methods
@@ -380,7 +380,7 @@ service Cassandra {
   3:required ColumnPath column_path, 
   4:required binary value, 
   5:required i64 timestamp, 
-  6:required ConsistencyLevel 
consistency_level=ConsistencyLevel.ONE)
+  6:required

[Cassandra Wiki] Update of HadoopSupport by jeremyhanna

2011-03-03 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HadoopSupport page has been changed by jeremyhanna.
The comment on this change is: Adding some more troubleshooting info in a 
separate section..
http://wiki.apache.org/cassandra/HadoopSupport?action=diffrev1=26rev2=27

--

   * [[#Pig|Pig]]
   * [[#Hive|Hive]]
   * [[#ClusterConfig|Cluster Configuration]]
+  * [[#Troubleshooting|Troubleshooting]]
   * [[#Support|Support]]
  
  Anchor(Overview)
@@ -37, +38 @@

  
   Hadoop Streaming 
  As of 0.7, there is support for 
[[http://hadoop.apache.org/common/docs/r0.20.0/streaming.html|Hadoop 
Streaming]].  For examples on how to use Streaming with Cassandra, see the 
contrib section of the Cassandra source.  The relevant tickets are 
[[https://issues.apache.org/jira/browse/CASSANDRA-1368|CASSANDRA-1368]] and 
[[https://issues.apache.org/jira/browse/CASSANDRA-1497|CASSANDRA-1497]].
- 
-  Some troubleshooting 
- Releases before  0.6.2/0.7 are affected by a small  resource leak that may 
cause jobs to fail (connections are not released  properly, causing a resource 
leak). Depending on your local setup you  may hit this issue, and workaround it 
by raising the limit of open file  descriptors for the process (e.g. in 
linux/bash using `ulimit -n 32000`).  The error will be reported on  the hadoop 
job side as a thrift !TimedOutException.
- 
- If you are testing the integration against a single node and you obtain  some 
failures, this may be normal: you are probably overloading the  single machine, 
which may again result in timeout errors. You can  workaround it by reducing 
the number of concurrent tasks
- 
- {{{
-  Configuration conf = job.getConfiguration();
-  conf.setInt(mapred.tasktracker.map.tasks.maximum,1);
- }}}
- Also, you may reduce the size in rows of the batch you  are reading from 
cassandra
- 
- {{{
-  ConfigHelper.setRangeBatchSize(job.getConfiguration(), 1000);
- }}}
  [[#Top|Top]]
  
  Anchor(Pig)
@@ -93, +79 @@

  
  [[#Top|Top]]
  
+ Anchor(Troubleshooting)
+ 
+ == Troubleshooting ==
+ If you are running into timeout exceptions, you might need to tweak one or 
both of these settings:
+  * '''cassandra.range.batch.size''' - the default is 4096, but you may need 
to lower this depending on your data.  This is either specified in your hadoop 
configuration or using 
`org.apache.cassandra.hadoop.ConfigHelper.setRangeBatchSize`.
+  * '''rpc_timeout_in_ms''' - this is set in your `cassandra.yaml` (in 0.6 
it's `RpcTimeoutInMillis` in `storage-conf.xml`).  The rpc timeout is not for 
timing out from the client but between nodes.  This can be increased to reduce 
chances of timing out.
+ 
+ Releases before 0.6.2/0.7 are affected by a small resource leak that may 
cause jobs to fail (connections are not released  properly, causing a resource 
leak). Depending on your local setup you may hit this issue, and workaround it 
by raising the limit of open file descriptors for the process (e.g. in 
linux/bash using `ulimit -n 32000`).  The error will be reported on the hadoop 
job side as a thrift !TimedOutException.
+ 
+ If you are testing the integration against a single node and you obtain some 
failures, this may be normal: you are probably overloading the single machine, 
which may again result in timeout errors. You can workaround it by reducing the 
number of concurrent tasks
+ 
+ {{{
+  Configuration conf = job.getConfiguration();
+  conf.setInt(mapred.tasktracker.map.tasks.maximum,1);
+ }}}
+ Also, you may reduce the size in rows of the batch you  are reading from 
cassandra
+ 
+ {{{
+  ConfigHelper.setRangeBatchSize(job.getConfiguration(), 1000);
+ }}}
+ 
+ [[#Top|Top]]
+ 
  Anchor(Support)
  
  == Support ==

[Cassandra Wiki] Trivial Update of HadoopSupport by jeremyhanna

2011-03-03 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HadoopSupport page has been changed by jeremyhanna.
http://wiki.apache.org/cassandra/HadoopSupport?action=diffrev1=27rev2=28

--

  
   Hadoop Streaming 
  As of 0.7, there is support for 
[[http://hadoop.apache.org/common/docs/r0.20.0/streaming.html|Hadoop 
Streaming]].  For examples on how to use Streaming with Cassandra, see the 
contrib section of the Cassandra source.  The relevant tickets are 
[[https://issues.apache.org/jira/browse/CASSANDRA-1368|CASSANDRA-1368]] and 
[[https://issues.apache.org/jira/browse/CASSANDRA-1497|CASSANDRA-1497]].
+ 
  [[#Top|Top]]
  
  Anchor(Pig)

[jira] Updated: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2269:
--

Attachment: 2269-0.6.txt

patch to extract exception logging from DTPE and call from the Thrift executor.

(the actual shutdown is done by the default exception hook we set up -- it's 
not normally called on executor threads because both Future and executor 
afterExecute machinery swallow exceptions.)

 OOM in the Thrift thread doesn't shut down server
 -

 Key: CASSANDRA-2269
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.6.13, 0.7.4

 Attachments: 2269-0.6.txt


 Example:
 {noformat}
 java.lang.OutOfMemoryError: Java heap space
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
 at 
 org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
 at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
 at 
 org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
 at 
 org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
 at 
 org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
 at 
 org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
 at 
 org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
 at 
 org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
 at 
 org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
 at 
 org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
 at 
 org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
 at 
 org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
 at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
 Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
 {noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076899 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/

Author: jbellis
Date: Fri Mar  4 00:16:57 2011
New Revision: 1076899

URL: http://svn.apache.org/viewvc?rev=1076899view=rev
Log:
merge from 0.6

Modified:
cassandra/branches/cassandra-0.7/   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

Propchange: cassandra/branches/cassandra-0.7/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Mar  4 00:16:57 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6:922689-1071777
+/cassandra/branches/cassandra-0.6:922689-1071777,1076891
 /cassandra/branches/cassandra-0.7:1026516,1035666,1050269
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Mar  4 00:16:57 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1071777
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1071777,1076891
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Mar  4 00:16:57 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1071777
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1071777,1076891
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Mar  4 00:16:57 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1071777
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1071777,1076891
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Mar  4 00:16:57 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1071777
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1071777,1076891
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1053690-1055654

[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002350#comment-13002350
 ] 

Jonathan Ellis commented on CASSANDRA-1969:
---

Those are never promoted to old gen, though, so the JVM can run the cleanup in 
the frequent and cheap new-gen collections.  Once you promote to old gen it 
gets much messier (because old gen collections can be days apart and take 
multiple seconds to finish).

 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-03-03 Thread Vijay (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002365#comment-13002365
 ] 

Vijay commented on CASSANDRA-1969:
--

Again coming back to the original POC code... where we will reuse the BB's 
which are not GC'ed (using weak references) shouldn't that approach work? (if 
we are not going to use JNA) i think that way we can guarantee to some extent 
that we dont fragment the Direct Memory... Assuming most of our queries results 
will be approx of the same size (where chunk size can be defined by the user).

 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1076905 - in /cassandra/branches/cassandra-0.7: CHANGES.txt contrib/pig/README.txt contrib/pig/bin/pig_cassandra contrib/pig/build.xml contrib/pig/src/java/org/apache/cassandra/hadoop/pig

2011-03-03 Thread brandonwilliams

Author: brandonwilliams
Date: Fri Mar  4 00:41:55 2011
New Revision: 1076905

URL: http://svn.apache.org/viewvc?rev=1076905view=rev
Log:
Pig storefunc.
Patch by brandonwilliams, reviewed by Jeremy Hanna for CASSANDRA-1828.

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/contrib/pig/README.txt
cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra
cassandra/branches/cassandra-0.7/contrib/pig/build.xml

cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076905r1=1076904r2=1076905view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Fri Mar  4 00:41:55 2011
@@ -2,6 +2,7 @@
  * add nodetool join command (CASSANDRA-2160)
  * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244)
  * initialize endpoing in gossiper earlier (CASSANDRA-2228)
+ * add ability to write to Cassandra from Pig (CASSANDRA-1828)
 
 0.7.3
  * Keep endpoint state until aVeryLongTime (CASSANDRA-2115)

Modified: cassandra/branches/cassandra-0.7/contrib/pig/README.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/README.txt?rev=1076905r1=1076904r2=1076905view=diff
==
--- cassandra/branches/cassandra-0.7/contrib/pig/README.txt (original)
+++ cassandra/branches/cassandra-0.7/contrib/pig/README.txt Fri Mar  4 00:41:55 
2011
@@ -1,4 +1,5 @@
-A Pig LoadFunc that reads all columns from a given ColumnFamily.
+A Pig storage class that reads all columns from a given ColumnFamily, or writes
+properly formatted results into a ColumnFamily.
 
 Setup:
 
@@ -7,10 +8,15 @@ configuration and set the PIG_HOME and J
 variables to the location of a Pig = 0.7.0 install and your Java
 install. 
 
+NOTE: if you intend to _output_ to Cassandra, until there is a Pig release that
+uses jackson  1.0.1 (see https://issues.apache.org/jira/browse/PIG-1863) you
+will need to build Pig yourself with jackson 1.4.  To do this, edit Pig's
+ivy/libraries.properties, and run ant.
+
 If you would like to run using the Hadoop backend, you should
 also set PIG_CONF_DIR to the location of your Hadoop config.
 
-FInally, set the following as environment variables (uppercase,
+Finally, set the following as environment variables (uppercase,
 underscored), or as Hadoop configuration variables (lowercase, dotted):
 * PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening on 
 * PIG_INITIAL_ADDRESS or cassandra.thrift.address : initial address to connect 
to
@@ -40,3 +46,11 @@ grunt namecounts = FOREACH namegroups G
 grunt orderednames = ORDER namecounts BY $0;
 grunt topnames = LIMIT orderednames 50;
 grunt dump topnames;
+
+Outputting to Cassandra requires the same format from input, so the simplest 
example is:
+
+grunt rows = LOAD 'cassandra://Keyspace1/Standard1' USING CassandraStorage();
+grunt STORE rows into 'cassandra://Keyspace1/Standard2' USING 
CassandraStorage();
+
+Which will copy the ColumnFamily.  Note that the destination ColumnFamily must
+already exist for this to work.

Modified: cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra?rev=1076905r1=1076904r2=1076905view=diff
==
--- cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra (original)
+++ cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra Fri Mar  4 
00:41:55 2011
@@ -38,7 +38,7 @@ if [ x$PIG_HOME = x ]; then
 fi
 
 # pig jar.
-PIG_JAR=$PIG_HOME/pig*core.jar
+PIG_JAR=$PIG_HOME/pig*.jar
 if [ ! -e $PIG_JAR ]; then
 echo Unable to locate Pig jar 2
 exit 1

Modified: cassandra/branches/cassandra-0.7/contrib/pig/build.xml
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/build.xml?rev=1076905r1=1076904r2=1076905view=diff
==
--- cassandra/branches/cassandra-0.7/contrib/pig/build.xml (original)
+++ cassandra/branches/cassandra-0.7/contrib/pig/build.xml Fri Mar  4 00:41:55 
2011
@@ -21,7 +21,7 @@
 !-- stores the environment for locating PIG_HOME --
 property environment=env /
 property name=cassandra.dir value=../.. /
-property name=cassandra.lib value= /
+property name=cassandra.lib value=${cassandra.dir}/lib /
 property name=cassandra.classes value=${cassandra.dir}/build/classes 
/
 property name=build.src value=${basedir}/src /
 property name=build.lib value=${basedir}/lib /
@@ -31,8 +31,9 @@
 
 path id=pig.classpath

[jira] Resolved: (CASSANDRA-1828) Create a pig storefunc


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-1828.
-

Resolution: Fixed

Committed with the second point change (also in the input format) and updated 
README

 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1828) Create a pig storefunc

2011-03-03 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002382#comment-13002382
 ] 

Hudson commented on CASSANDRA-1828:
---

Integrated in Cassandra-0.7 #345 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/345/])
Pig storefunc.
Patch by brandonwilliams, reviewed by Jeremy Hanna for CASSANDRA-1828.


 Create a pig storefunc
 --

 Key: CASSANDRA-1828
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1828
 Project: Cassandra
  Issue Type: New Feature
  Components: Contrib, Hadoop
Affects Versions: 0.7 beta 1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, 
 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt

   Original Estimate: 32h
  Remaining Estimate: 32h

 Now that we have a ColumnFamilyOutputFormat, we can write data back to 
 cassandra in mapreduce jobs, however we can only do this in java.  It would 
 be nice if pig could also output to cassandra.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2270) nodetool info NPE when node isn't fully booted

2011-03-03 Thread JIRA

nodetool info NPE when node isn't fully booted
--

 Key: CASSANDRA-2270
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2270
 Project: Cassandra
  Issue Type: Bug
Reporter: Sébastien Giroux
Priority: Minor
 Fix For: 0.7.4


Running nodetool -h 127.0.0.1 info when the node is not yet ready throw a NPE.

Exception in thread main java.lang.NullPointerException
at 
org.apache.cassandra.gms.Gossiper.getCurrentGenerationNumber(Gossiper.java:313)
at 
org.apache.cassandra.service.StorageService.getCurrentGenerationNumber(StorageService.java:1239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
at 
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
at 
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at 
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:205)


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2027) term definitions

2011-03-03 Thread Eric Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002396#comment-13002396
 ] 

Eric Evans commented on CASSANDRA-2027:
---


The following reboot is the result of a discussion between Gary Dusbabek, 
Jonathan Ellis, and myself (any errors or misunderstandings are my fault).

h2. Revised definitions (+ semantics)

h3. String

Anything between quotes, _or_ any unquoted alnum values that begins with a 
letter.

Examples:

{code:style=SQL}
SELECT 0day FROM cf;
SELECT B_day FROM cf;

UPDATE cf SET value-low = 14% WHERE KEY = @skinny;
UPDATE cf SET foo = bar WHERE KEY = baz;
{code}

h3. Integer

An undecorated numeric literal.  How the term is converted node-side is 
determined by the comparator/validator in use.  For example, {{100}} could be 
converted to a 4-byte integer or an 8-byte long depending on whether the 
comparator/validator was an {{IntegerType}} or {{LongType}} respectively.

Examples:

{code:style=SQL}
SELECT 10..100 FROM cf WHERE KEY = key;
UPDATE cf SET 1000 = thousand, 100 = hundred WHERE KEY = key;
{code}

h3. UUID

A UUID formated as a hexidecimal-hyphenated string (i.e. 
{{b137dd10-45b6-11e0-8955-00247ee1f924}}).

Examples:

{code:style=SQL}
SELECT f1fa6c22-45b7-11e0-8955-00247ee1f924 FROM cf WHERE KEY = key;
UPDATE cf SET 0ceb632e-45b8-11e0-8955-00247ee1f924 = 9 WHERE KEY = key;
{code}

As a special-case, when the comparator/validator is TimeUUIDType, a quoted 
string literal can be used to supply a parse-able timestamp (currently most 
ISO8601 variants).

{code:style=SQL}
SELECT 2011-01-01..2011-02-01 FROM cf WHERE KEY = key;
{code}

_Note: it doesn't make sense to try to query by-column using a timestamp like 
this, because date-time is only one component of a type 1 UUID.  The docs will 
need to be clear about this._

h3. UTF-8

A double-quoted string literal that is prefixed with a u to indicate that it 
should be encoded to bytes using the utf-8 charset node-side.

Examples:

{code:style=SQL}
SELECT uname FROM cf;
UPDATE cf SET uname = uvalue WHERE KEY = key;
{code}

_This one is iffy. Consensus seems to be that the UTF8 charset should 
implicitly be used in the conversion to bytes when comparator/validator is 
UTF8Type. If that's the case, then the only time where this term would do 
anything useful would be for storing UTF8 where comparator/validator is 
BytesType. That seems corner-case enough for me to warrant leaving it out 
entirely._



One point of contention during the discussion that spawned this reboot was type 
inference.  What's proposed above adds some inference, (namely for unicode, 
decimal values, and some UUID cases), but I'm going to make one more attempt at 
stopping it there. I'm nothing if not persistent, right? :)

For example, Least Surprise says that {{10}} and {{10}} differ in that one is 
explicitly a string, so converting it to a numeric type with a decimal value of 
10 (still) seems wrong to me.  I'd prefer to raise an exception for such 
mismatches, which also seems like a good way of protecting users from a whole 
class of bugs.

I'm also continuing to have a hard time accepting that different rules should 
exist (syntax and semantics) for column names and values. The general argument 
for SQL parity is a strong one, and I'm trying to be convinced on this issue, 
(honest), but I keep coming back to the notion that SQL column names are not 
typed, and that forcing that distinction on Cassandra seems contrived.

 term definitions
 

 Key: CASSANDRA-2027
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2027
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: v1-0001-CASSANDRA-2027-utf8-and-integer-term-types.txt, 
 v1-0002-column-name-validation.txt, 
 v1-0003-system-tests-for-integer-and-utf8-term-types.txt, 
 v1-0004-uuid-term-definitions.txt, 
 v1-0005-missed-doc-update-for-utf8-term-type.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 h3. String
 Anything between double-quotes.  Node-side this is just converted to bytes, 
 so it could really be used to represent *any* type so long as it is 
 appropriately encoded.
 Examples:
 {code:style=SQL}
 SELECT name FROM cf;
 UPDATE cf SET name = value WHERE KEY = key;
 {code}
 h3. UTF-8
 A double-quoted string literal that is prefixed with a u to indicated that 
 it should be encoded to bytes using the utf-8 charset node-side.
 Examples:
 {code:style=SQL}
 SELECT uname FROM cf;
 UPDATE cf SET uname = uvalue WHERE KEY = key;
 {code}
 h3. Integer
 An undecorated numeric literal, converted to a 4-byte int node-side.
 Examples:
 {code:style=SQL}
 SELECT 10..100 FROM cf WHERE KEY = key;

[jira] Commented: (CASSANDRA-2252) off-heap memtables

2011-03-03 Thread Stu Hood (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002414#comment-13002414
 ] 

Stu Hood commented on CASSANDRA-2252:
-

I'd like to go ahead and merge these patches together onto jbellis's version. 
Things that I will cherry pick from the alternate patch:
* A generic Allocator interface, rather than referring to Memtables. There are 
various other places that we need to apply slabbing
* The additional counter tests we added
* Using the slab allocator's allocation count to determine throughput for 
memtables



The other realization we've had about slab allocation is that unless _all_ 
sources of fragmentation are eliminated, slabbing actually causes promotion 
failures to happen earlier, since it is harder to promote a slab into a 
fragmented oldgen. The other sources of fragmentation we suspect are:
* IndexSummaries (easily slabbed)
* the key and row caches (row cache tackled in CASSANDRA-1969)

These can probably be defragged in separate tickets, as long as we commit to 
fixing them.

 off-heap memtables
 --

 Key: CASSANDRA-2252
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 0.8

 Attachments: 0001-add-MemtableAllocator.txt, 
 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz


 The memtable design practically actively fights Java's GC design.  Todd 
 Lipcon gave a good explanation over on HBASE-3455.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Issue Comment Edited: (CASSANDRA-2252) off-heap memtables

2011-03-03 Thread Stu Hood (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002414#comment-13002414
 ] 

Stu Hood edited comment on CASSANDRA-2252 at 3/4/11 3:18 AM:
-

I'd like to go ahead and merge these patches together onto jbellis's version. 
Things that I will cherry pick from the alternate patch:
* A generic Allocator interface, rather than referring to Memtables. There are 
various other places that we need to apply slabbing
* The additional counter tests we added
* Using the slab allocator's allocation count to determine throughput for 
memtables
* Slabbing of keys



The other realization we've had about slab allocation is that unless _all_ 
sources of fragmentation are eliminated, slabbing actually causes promotion 
failures to happen earlier, since it is harder to promote a slab into a 
fragmented oldgen. The other sources of fragmentation we suspect are:
* IndexSummaries (easily slabbed)
* the key and row caches (row cache tackled in CASSANDRA-1969)

These can probably be defragged in separate tickets, as long as we commit to 
fixing them.

  was (Author: stuhood):
I'd like to go ahead and merge these patches together onto jbellis's 
version. Things that I will cherry pick from the alternate patch:
* A generic Allocator interface, rather than referring to Memtables. There are 
various other places that we need to apply slabbing
* The additional counter tests we added
* Using the slab allocator's allocation count to determine throughput for 
memtables



The other realization we've had about slab allocation is that unless _all_ 
sources of fragmentation are eliminated, slabbing actually causes promotion 
failures to happen earlier, since it is harder to promote a slab into a 
fragmented oldgen. The other sources of fragmentation we suspect are:
* IndexSummaries (easily slabbed)
* the key and row caches (row cache tackled in CASSANDRA-1969)

These can probably be defragged in separate tickets, as long as we commit to 
fixing them.
  
 off-heap memtables
 --

 Key: CASSANDRA-2252
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 0.8

 Attachments: 0001-add-MemtableAllocator.txt, 
 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz


 The memtable design practically actively fights Java's GC design.  Todd 
 Lipcon gave a good explanation over on HBASE-3455.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2252) off-heap memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002440#comment-13002440
 ] 

Jonathan Ellis commented on CASSANDRA-2252:
---

sounds reasonable

 off-heap memtables
 --

 Key: CASSANDRA-2252
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 0.8

 Attachments: 0001-add-MemtableAllocator.txt, 
 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz


 The memtable design practically actively fights Java's GC design.  Todd 
 Lipcon gave a good explanation over on HBASE-3455.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1077486 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java

Author: jbellis
Date: Fri Mar  4 04:19:58 2011
New Revision: 1077486

URL: http://svn.apache.org/viewvc?rev=1077486view=rev
Log:
javadoc for BBU.clone

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java?rev=1077486r1=1077485r2=1077486view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
 Fri Mar  4 04:19:58 2011
@@ -188,23 +188,28 @@ public class ByteBufferUtil
throw new RuntimeException(e);
 } 
 }
-
-public static ByteBuffer clone(ByteBuffer o)
+
+/**
+ * @return a new copy of the data in @param buffer
+ * USUALLY YOU SHOULD USE ByteBuffer.duplicate() INSTEAD, which creates a 
new Buffer
+ * (so you can mutate its position without affecting the original) without 
copying the underlying array.
+ */
+public static ByteBuffer clone(ByteBuffer buffer)
 {
-assert o != null;
+assert buffer != null;
 
-if (o.remaining() == 0)
+if (buffer.remaining() == 0)
 return EMPTY_BYTE_BUFFER;
   
-ByteBuffer clone = ByteBuffer.allocate(o.remaining());
+ByteBuffer clone = ByteBuffer.allocate(buffer.remaining());
 
-if (o.hasArray())
+if (buffer.hasArray())
 {
-System.arraycopy(o.array(), o.arrayOffset() + o.position(), 
clone.array(), 0, o.remaining());
+System.arraycopy(buffer.array(), buffer.arrayOffset() + 
buffer.position(), clone.array(), 0, buffer.remaining());
 }
 else
 {
-clone.put(o.duplicate());
+clone.put(buffer.duplicate());
 clone.flip();
 }

[jira] Created: (CASSANDRA-2271) Audit uses of BBU.clone

Audit uses of BBU.clone
---

 Key: CASSANDRA-2271
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2271
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.4
 Attachments: 2271.txt



-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2271) Audit uses of BBU.clone