Hi, We got an issue here with cassandra 1.1.6 where a secondary index seems to be seen as empty. For example, If I try to see what are the first 10 values I have :
cqlsh:pns_fr> select mailendwnd from syndic limit 10; mailendwnd -------------------------- 2012-11-29 23:30:00+0000 2012-11-29 02:35:00+0000 2012-11-29 06:35:00+0000 2012-11-29 00:30:00+0000 2012-11-29 07:10:00+0000 2012-11-29 02:10:00+0000 2012-11-29 01:55:00+0000 2012-11-29 23:45:00+0000 2012-11-29 07:25:00+0000 2012-11-29 06:55:00+0000 However, even if there is a secondary index on mailendwnd, we can't get any record : cqlsh:pns_fr> select mailendwnd from syndic where mailendwnd = '2012-11-29 02:35:00+0000' limit 1000; fyi, It's resolved by repairing the index and it happened a few times. I launched cassandra in debug mode and noticed that it thinks there is no data in the index : DEBUG [Thrift:14] 2012-11-30 08:35:20,756 CassandraServer.java (line 1232) execute_cql_query DEBUG [Thrift:14] 2012-11-30 08:35:20,758 QueryProcessor.java (line 445) CQL statement type: SELECT DEBUG [Thrift:14] 2012-11-30 08:35:20,777 StorageProxy.java (line 842) Command/ConsistencyLevel is RangeSliceCommand{keyspace='pns_fr', column_family='syndic', super_column=null, predicate=SlicePredicate(column_names:[java.nio.HeapByteBuffer[pos=0 lim=10 cap=10]]), range=[min(-1),m in(-1)], row_filter =[IndexExpression(column_name:6D 61 69 6C 65 6E 64 77 6E 64, op:EQ, value:00 00 01 3B 4A 04 64 20)], maxResults=10000, maxIsColumns=false}/ONE DEBUG [Thrift:14] 2012-11-30 08:35:20,778 StorageProxy.java (line 1073) restricted ranges for query [min(-1),min(-1)] are [[min(-1),max(0)], (max(0),max(21267647932558653966460912964485513216)], (max(21267647932558653966460912964485513216),max(42535295865117307932921825928971026432 )], (max(42535295865117307932921825928971026432),max(63802943797675961899382738893456539648)], (max(63802943797675961899382738893456539648),max(85070591730234615865843651857942052864)], (max(85070591730234615865843651857942052864),max(106338239662793269832304564822427566080)], (max (106338239662793269832304564822427566080),max(127605887595351923798765477786913079296)], (max(127605887595351923798765477786913079296),max(148873535527910577765226390751398592512)], (max(148873535527910577765226390751398592512),min(-1)]] DEBUG [Thrift:14] 2012-11-30 08:35:20,779 NetworkTopologyStrategy.java (line 125) /10.244.136.105,/10.244.137.238,/10.244.130.226 endpoints in datacenter us-west-2 for token 0 DEBUG [Thrift:14] 2012-11-30 08:35:20,780 StorageProxy.java (line 869) local range slice DEBUG [Thrift:14] 2012-11-30 08:35:20,782 ExtendedFilter.java (line 181) adding columns to original Filter to cover additional expressions DEBUG [Thrift:14] 2012-11-30 08:35:20,782 KeysSearcher.java (line 99) Primary scan clause is mailendwnd DEBUG [Thrift:14] 2012-11-30 08:35:20,783 ColumnFamilyStore.java (line 1449) Filtering org.apache.cassandra.db.index.keys.KeysSearcher$1@60ce274e for rows matching org.apache.cassandra.db.filter.ExtendedFilter$FilterWithClauses@7e76a388 DEBUG [Thrift:14] 2012-11-30 08:35:20,784 KeysSearcher.java (line 135) Scanning index 'syndic.mailendwnd EQ 2012-11-29 02:35:00+0000' starting with DEBUG [Thrift:14] 2012-11-30 08:35:20,784 CollationController.java (line 227) collectAllData DEBUG [Thrift:14] 2012-11-30 08:35:20,784 KeysSearcher.java(line 145) fetched null DEBUG [Thrift:14] 2012-11-30 08:35:20,784 KeysSearcher.java (line 148) no data, all done <----- THIS IS WHAT IT SAYS HERE I tried to remotely debug cassandra from eclipse but I wasn't able to do it. Whenever it tries to connect to the dt_socket it fails and the port is closed on the server :( I tried locally without any issue but I havent the data (can I copy data from one node to mine and debug it locally, is it possible ?). I'm investigating network issues concerning that to help you. If you have input concerning the issue before I can debug it, welcome ! Regards Cyril SCETBON