Re: Secondary index data gone after restart (1.1.1)

2012-06-26 Thread Fei Shan 
Hi

   please refer JDK nio package's ByteBuffer, I don't think that ByteBuffer
can be cast from the BigInteger directly,
it seems you need make some conversion before put it into a ByteBuffer.

Thanks
Fei

On Tue, Jun 26, 2012 at 12:07 AM, Ivo Meißner i...@overtronic.com wrote:

 Hi,

 I am running into some problems with secondary indexes that I am unable to
 track down. When I restart the cassandra service, the secondary index data
 won't load and I get the following error during startup:

 INFO 08:29:42,127 Opening
 /var/myproject/cassandra/data/mykeyspace/group_admin/mykeyspace-group_admin.group_admin_groupId_idx-hd-1
 (20808 bytes)
 ERROR 08:29:42,159 Exception in thread Thread[SSTableBatchOpen:1,5,main]
 java.lang.ClassCastException: java.math.BigInteger cannot be cast to
 java.nio.ByteBuffer
 at
 org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:37)
 at org.apache.cassandra.dht.LocalToken.compareTo(LocalToken.java:45)
 at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:89)
 at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:38)
 at java.util.TreeMap.getEntry(TreeMap.java:328)
 at java.util.TreeMap.containsKey(TreeMap.java:209)
 at java.util.TreeSet.contains(TreeSet.java:217)
 at
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:396)
 at
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:187)
 at
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:225)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)

 When the service starts I can still select data from the column family,
 but not using the secondary index.
 After I execute nodetool rebuild_index the secondary index works fine
 again until the next restart.

 The error only seems to occur on the column groupId (TimeUUIDType). The
 other index on userId seems to work.

 I have the following column family definition:

 create column family group_admin with
   comparator = UTF8Type and
   key_validation_class = UTF8Type and
   column_metadata = [
 {column_name: id, validation_class: UTF8Type},
 {column_name: added, validation_class: LongType},
 {column_name: userId, validation_class: BytesType, index_type: KEYS},
 {column_name: requestMessage, validation_class: UTF8Type},
 {column_name: status, validation_class: LongType},
 {column_name: groupId, validation_class: TimeUUIDType, index_type:
 KEYS}
   ];

 Thank you very much for your help!

 Ivo

Re: store large String as col value

2012-06-20 Thread Fei Shan 
why not just store the URL of pic / video, just like the RDBMS did,
cassandra is merely a distributed database from some kind point of view. it
is more like a schema design issue. Any video or pic should not be put into
the DB especially big data table DB as raw binary format. and did this kind
of thing gonna cause the performance issue afterward, since it
is against the original design of cassandra.

On Wed, Jun 20, 2012 at 2:22 PM, Paolo Bernardi berna...@gmail.com wrote:

 Take also into account Thrift's limits.

 http://wiki.apache.org/cassandra/CassandraLimitations

 Paolo
 On Jun 20, 2012 10:11 PM, Dave Brosius dbros...@mebigfatguy.com wrote:


 Column values are limited at 2G.

 Why store them as Base64? that just adds overhead. Storing the raw bytes
 will save you a bunch.


  *- Original Message -*
 *From:* Cyril Auburtin cyril.aubur...@gmail.com
 *Sent:* Wed, June 20, 2012 13:38
 *Subject:* store large String as col value

 In order to store picture, transfered as a base 64 encoded String in the
 message

 I wanted to ask, what is the limit for column values (since the wiki is
 down just now)

 the row has a limit of 2GB, it should be enough for pictures, but a
 single column Row, has als o a limit of ~2GB?

 Is it critical with replication? Is it better to cut down the String in
 shorter pieces and store them in different rows to support more easily
 read/write towards replication?