[
https://issues.apache.org/jira/browse/CASSANDRA-4532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449764#comment-13449764
]
basanth gowda edited comment on CASSANDRA-4532 at 9/7/12 3:00 AM:
------------------------------------------------------------------
Sylvian,
Thanks for the fix, May be I should have opened the ticket differently. My main
issue is not that the connection on cqlsh was getting closed (because of NPE)
or exception being thrown in thrift client. To adopt CQL3 mainstream slice with
paging is need, I saw some other ticket, where this proposal was there.
Main reason I opened the ticket was for not being able to slice (rather no
syntax support for that, to continue where the last query left off). This is a
piece of code that was written using Hector and my assumption is the filtering
was done on server side. We are using this in production. I modified attributes
(to get rid of proprietary stuff, so may be broken, but should give an idea of
what I was trying to do with CQL3)
long cStartTime = 0L;
long startTime = System.nanoTime();
boolean fetchNextBatch = true;
int totalKeysFetched = 0;
String lastKeyFetched = null;
long cEndTime = <Some Time in Millis>;
while(fetchNextBatch) {
fetchNextBatch = false;
int pageSize = 3; //Just for demonstration
SliceQuery<Object,DynamicComposite,String> sliceQuery =
HFactory.createSliceQuery(keyspace,<KEY
SERIALIZER>,DynamicCompositeSerializer.get(),StringSerializer.get());
sliceQuery.setKey("C");
sliceQuery.setColumnFamily("<SOME CF NAME>");
DynamicComposite startRange = new DynamicComposite();
startRange.addComponent(cStartTime,LongSerializer.get()); //For
the first fetch - this will be 0L
startRange.addComponent(lastKeyFetched,StringSerializer.get());
// this will be null for first fetch
DynamicComposite endRange = new DynamicComposite();
endRange.addComponent(new Long(cEndTime), LongSerializer.get(),
"LongType", AbstractComposite.ComponentEquality.LESS_THAN_EQUAL);
//Add another config if we need columnPageSize
sliceQuery.setRange(startRange,endRange,false,pageSize);
long start = System.nanoTime();
QueryResult<ColumnSlice<DynamicComposite, String>> result =
sliceQuery.execute();
float t = (float)((System.nanoTime() - start)/1000000);
System.out.println("TIME FOR QUERY :" + t + " MILLI SECONDS");
ColumnSlice<DynamicComposite, String> cs = result.get();
List<HColumn<DynamicComposite,String>> compositeList =
cs.getColumns();
for(int i =0;i<compositeList.size();i++) {
HColumn<DynamicComposite, String> col =
compositeList.get(i);
cStartTime = col.getName().get(0,LongSerializer.get());
//This will be the cTime for the start range of the next query
lastKeyFetched =
col.getName().get(1,StringSerializer.get()); //In the start range for the next
query, this key will be used.
keyTimeMap.put(lastKeyFetched,scheduleStartTime);
totalKeysFetched ++;
}
//Process Fetched Data
fetchNextBatch = compositeList.size() == pageSize; // If the
number of records retrieved is equal to the page size, then there are probably
more records left
}
was (Author: basu76):
Sylvian,
Thanks for the fix, May be I should have opened the ticket differently. My main
issue is not that the connection on cqlsh was getting closed (because of NPE)
or exception being thrown in thrift client. To adopt CQL3 mainstream slice with
paging is need, I saw some other ticket, where this proposal was there.
Main reason I opened the ticket was for not being able to slice (rather no
syntax support for that, to continue where the last query left off). This is a
piece of code that was written using Hector and my assumption is the filtering
was done on server side. We are using this in production. I modified attributes
(to get rid of proprietary stuff, so may be broken, but should give an idea of
what I was trying to do with CQL3)
long cStartTime = 0L;
long startTime = System.nanoTime();
boolean fetchNextBatch = true;
int totalKeysFetched = 0;
String lastKeyFetched = null;
long cEndTime = <Some Time in Millis>;
while(fetchNextBatch) {
fetchNextBatch = false;
int pageSize = 3; //Just for demonstration
SliceQuery<Object,DynamicComposite,String> sliceQuery =
HFactory.createSliceQuery(keyspace,cassCFConfig.getKeySerializer(),DynamicCompositeSerializer.get(),StringSerializer.get());
sliceQuery.setKey("C");
sliceQuery.setColumnFamily("<SOME CF NAME>");
DynamicComposite startRange = new DynamicComposite();
startRange.addComponent(cStartTime,LongSerializer.get()); //For
the first fetch - this will be 0L
startRange.addComponent(lastKeyFetched,StringSerializer.get());
// this will be null for first fetch
DynamicComposite endRange = new DynamicComposite();
endRange.addComponent(new Long(cEndTime), LongSerializer.get(),
"LongType", AbstractComposite.ComponentEquality.LESS_THAN_EQUAL);
//Add another config if we need columnPageSize
sliceQuery.setRange(startRange,endRange,false,pageSize);
long start = System.nanoTime();
QueryResult<ColumnSlice<DynamicComposite, String>> result =
sliceQuery.execute();
float t = (float)((System.nanoTime() - start)/1000000);
System.out.println("TIME FOR QUERY :" + t + " MILLI SECONDS");
ColumnSlice<DynamicComposite, String> cs = result.get();
List<HColumn<DynamicComposite,String>> compositeList =
cs.getColumns();
for(int i =0;i<compositeList.size();i++) {
HColumn<DynamicComposite, String> col =
compositeList.get(i);
cStartTime = col.getName().get(0,LongSerializer.get());
//This will be the cTime for the start range of the next query
lastKeyFetched =
col.getName().get(1,StringSerializer.get()); //In the start range for the next
query, this key will be used.
keyTimeMap.put(lastKeyFetched,scheduleStartTime);
totalKeysFetched ++;
}
//Process Fetched Data
fetchNextBatch = compositeList.size() == pageSize; // If the
number of records retrieved is equal to the page size, then there are probably
more records left
}
> NPE when trying to select a slice from a composite table
> --------------------------------------------------------
>
> Key: CASSANDRA-4532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4532
> Project: Cassandra
> Issue Type: Bug
> Components: API, Core
> Affects Versions: 1.1.3
> Environment: Cassandra 1.1.3 (2 nodes) on a single host - mac osx
> Reporter: basanth gowda
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: Slice, cql, cql3
> Fix For: 1.1.6
>
> Attachments: 4532.txt
>
>
> I posted this question on StackOverflow, because i need a solution.
> Created a table with :
> {noformat}
> create table compositetest(m_id ascii,i_id int,l_id ascii,body ascii, PRIMARY
> KEY(m_id,i_id,l_id));
> {noformat}
> wanted to slice the results returned, so did something like below, not sure
> if its the right way. The first one returns data perfectly as expected,
> second one to get the next 3 columns closes the transport of my cqlsh
> {noformat}
> cqlsh:testkeyspace1> select * from compositetest where i_id<=3 limit 3;
> m_id | i_id | l_id | body
> ------+------+------+------
> m1 | 1 | l1 | b1
> m1 | 2 | l2 | b2
> m2 | 1 | l1 | b1
> cqlsh:testkeyspace1> Was trying to write something for slice range.
> TSocket read 0 bytes
> {noformat}
> Is there a way to achieve what I am doing here, it would be good if some
> meaning ful error is sent back, instead of cqlsh closing the transport.
> On the server side I see the following error.
> {noformat}
> ERROR [Thrift:3] 2012-08-12 15:15:24,414 CustomTThreadPoolServer.java (line
> 204) Error occurred during processing of message.
> java.lang.NullPointerException
> at
> org.apache.cassandra.cql3.statements.SelectStatement$Restriction.setBound(SelectStatement.java:1277)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.updateRestriction(SelectStatement.java:1151)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:1001)
> at
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:215)
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:121)
> at
> org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1237)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3542)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3530)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {noformat}
> With ThriftClient I get :
> {noformat}
> org.apache.thrift.transport.TTransportException
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql_query(Cassandra.java:1402)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql_query(Cassandra.java:1388)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
