[
https://issues.apache.org/jira/browse/CASSANDRA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015932#comment-13015932
]
Tey Kar Shiang edited comment on CASSANDRA-2401 at 4/6/11 10:15 AM:
--------------------------------------------------------------------
hi, yes. it seems to me so. Here, we create a table "FileMap", in which we
store columns e.g. "content", "authorID", "Version", "Modified Time", "File
Type", etc. Among them, sorted indices are "authorID" (as UserIndex), "File
Type", "Modified Time", and "CassType"; where CassType means generally 'file
type' here in our case. It is not used though.
{{03/30/2011 09:34 AM 11,366,878 FileMap-f-53-Data.db}}
{{03/30/2011 09:34 AM 78,496 FileMap-f-53-Filter.db}}
{{03/30/2011 09:34 AM 735,930 FileMap-f-53-Index.db}}
{{03/30/2011 09:34 AM 4,264 FileMap-f-53-Statistics.db}}
{{03/30/2011 05:37 PM 4,055 FileMap-f-54-Data.db}}
{{03/30/2011 05:37 PM 40 FileMap-f-54-Filter.db}}
{{03/30/2011 05:37 PM 270 FileMap-f-54-Index.db}}
{{03/30/2011 05:37 PM 4,264 FileMap-f-54-Statistics.db}}
{{04/04/2011 04:07 PM 24,068 FileMap-f-55-Data.db}}
{{04/04/2011 04:07 PM 200 FileMap-f-55-Filter.db}}
{{04/04/2011 04:07 PM 1,746 FileMap-f-55-Index.db}}
{{04/04/2011 04:07 PM 4,264 FileMap-f-55-Statistics.db}}
{{04/04/2011 04:07 PM 961,808 FileMap.CassTypeIndex-f-53-Data.db}}
{{04/04/2011 04:07 PM 1,936 FileMap.CassTypeIndex-f-53-Filter.db}}
{{04/04/2011 04:07 PM 11 FileMap.CassTypeIndex-f-53-Index.db}}
{{04/04/2011 04:07 PM 4,264
FileMap.CassTypeIndex-f-53-Statistics.db}}
{{03/29/2011 02:52 PM 961,386 FileMap.FileTypeIndex-f-50-Data.db}}
{{03/29/2011 02:52 PM 1,936 FileMap.FileTypeIndex-f-50-Filter.db}}
{{03/29/2011 02:52 PM 11 FileMap.FileTypeIndex-f-50-Index.db}}
{{03/29/2011 02:52 PM 4,264
FileMap.FileTypeIndex-f-50-Statistics.db}}
{{03/30/2011 05:37 PM 404 FileMap.FileTypeIndex-f-51-Data.db}}
{{03/30/2011 05:37 PM 16 FileMap.FileTypeIndex-f-51-Filter.db}}
{{03/30/2011 05:37 PM 11 FileMap.FileTypeIndex-f-51-Index.db}}
{{03/30/2011 05:37 PM 4,264
FileMap.FileTypeIndex-f-51-Statistics.db}}
{{04/04/2011 04:07 PM 2,358 FileMap.FileTypeIndex-f-52-Data.db}}
{{04/04/2011 04:07 PM 16 FileMap.FileTypeIndex-f-52-Filter.db}}
{{04/04/2011 04:07 PM 11 FileMap.FileTypeIndex-f-52-Index.db}}
{{04/04/2011 04:07 PM 4,264
FileMap.FileTypeIndex-f-52-Statistics.db}}
{{03/29/2011 02:52 PM 3,298,947 FileMap.ModifiedIndex-f-50-Data.db}}
{{03/29/2011 02:52 PM 78,016 FileMap.ModifiedIndex-f-50-Filter.db}}
{{03/29/2011 02:52 PM 731,106 FileMap.ModifiedIndex-f-50-Index.db}}
{{03/29/2011 02:52 PM 4,264
FileMap.ModifiedIndex-f-50-Statistics.db}}
{{03/30/2011 05:37 PM 2,065 FileMap.ModifiedIndex-f-51-Data.db}}
{{03/30/2011 05:37 PM 64 FileMap.ModifiedIndex-f-51-Filter.db}}
{{03/30/2011 05:37 PM 450 FileMap.ModifiedIndex-f-51-Index.db}}
{{03/30/2011 05:37 PM 4,264
FileMap.ModifiedIndex-f-51-Statistics.db}}
{{04/04/2011 04:07 PM 13,835 FileMap.ModifiedIndex-f-52-Data.db}}
{{04/04/2011 04:07 PM 328 FileMap.ModifiedIndex-f-52-Filter.db}}
{{04/04/2011 04:07 PM 3,006 FileMap.ModifiedIndex-f-52-Index.db}}
{{04/04/2011 04:07 PM 4,264
FileMap.ModifiedIndex-f-52-Statistics.db}}
{{04/04/2011 04:07 PM 962,874 FileMap.UserIndex-f-53-Data.db}}
{{04/04/2011 04:07 PM 1,936 FileMap.UserIndex-f-53-Filter.db}}
{{04/04/2011 04:07 PM 420 FileMap.UserIndex-f-53-Index.db}}
{{04/04/2011 04:07 PM 4,264 FileMap.UserIndex-f-53-Statistics.db}}
In the search, we are using IndexClause as:
ByteBuffer field_author = ByteBuffer.wrap(new byte[]{'a'});
ByteBuffer author_1 = IntegerSerializer.get().toByteBuffer(1);
ByteBuffer file_type = ByteBuffer.wrap(new byte[]{'t'});
ByteBuffer filetype_3 = ByteBuffer.wrap(new byte[]{3}); //file
type 3
IndexClause indexClause = new IndexClause();
indexClause.setCount(3000);
ArrayList<IndexExpression> expressions = new ArrayList();
expressions.add(new IndexExpression(field_author,
IndexOperator.EQ, author_1)); //user ID = 1
expressions.add(new IndexExpression(file_type,
IndexOperator.EQ, filetype_3)); //file type = 3
indexClause.setExpressions(expressions);
indexClause.setStart_key(new byte[]{});
}}
In the search, it scans all the indices from "FileMap.UserIndex", within which
there seems having a key (index) which is not found in the table "FileMap"; and
I roughly get that it breaks at data retrieval with "FileMap-f-53-Data", when
the position for the key is not found / available in "FileMap-f-53-Data".
was (Author: karshiang):
hi, yes. it seems to me so. Here, we create a table "FileMap", in which we
store columns e.g. "content", "authorID", "Version", "Modified Time", "File
Type", etc. Among them, sorted indices are "authorID" (as UserIndex), "File
Type", "Modified Time", and "CassType"; where CassType means generally 'file
type' here in our case. It is not used though.
{{
03/30/2011 09:34 AM 11,366,878 FileMap-f-53-Data.db
03/30/2011 09:34 AM 78,496 FileMap-f-53-Filter.db
03/30/2011 09:34 AM 735,930 FileMap-f-53-Index.db
03/30/2011 09:34 AM 4,264 FileMap-f-53-Statistics.db
03/30/2011 05:37 PM 4,055 FileMap-f-54-Data.db
03/30/2011 05:37 PM 40 FileMap-f-54-Filter.db
03/30/2011 05:37 PM 270 FileMap-f-54-Index.db
03/30/2011 05:37 PM 4,264 FileMap-f-54-Statistics.db
04/04/2011 04:07 PM 24,068 FileMap-f-55-Data.db
04/04/2011 04:07 PM 200 FileMap-f-55-Filter.db
04/04/2011 04:07 PM 1,746 FileMap-f-55-Index.db
04/04/2011 04:07 PM 4,264 FileMap-f-55-Statistics.db
04/04/2011 04:07 PM 961,808 FileMap.CassTypeIndex-f-53-Data.db
04/04/2011 04:07 PM 1,936 FileMap.CassTypeIndex-f-53-Filter.db
04/04/2011 04:07 PM 11 FileMap.CassTypeIndex-f-53-Index.db
04/04/2011 04:07 PM 4,264 FileMap.CassTypeIndex-f-53-Statistics.db
03/29/2011 02:52 PM 961,386 FileMap.FileTypeIndex-f-50-Data.db
03/29/2011 02:52 PM 1,936 FileMap.FileTypeIndex-f-50-Filter.db
03/29/2011 02:52 PM 11 FileMap.FileTypeIndex-f-50-Index.db
03/29/2011 02:52 PM 4,264 FileMap.FileTypeIndex-f-50-Statistics.db
03/30/2011 05:37 PM 404 FileMap.FileTypeIndex-f-51-Data.db
03/30/2011 05:37 PM 16 FileMap.FileTypeIndex-f-51-Filter.db
03/30/2011 05:37 PM 11 FileMap.FileTypeIndex-f-51-Index.db
03/30/2011 05:37 PM 4,264 FileMap.FileTypeIndex-f-51-Statistics.db
04/04/2011 04:07 PM 2,358 FileMap.FileTypeIndex-f-52-Data.db
04/04/2011 04:07 PM 16 FileMap.FileTypeIndex-f-52-Filter.db
04/04/2011 04:07 PM 11 FileMap.FileTypeIndex-f-52-Index.db
04/04/2011 04:07 PM 4,264 FileMap.FileTypeIndex-f-52-Statistics.db
03/29/2011 02:52 PM 3,298,947 FileMap.ModifiedIndex-f-50-Data.db
03/29/2011 02:52 PM 78,016 FileMap.ModifiedIndex-f-50-Filter.db
03/29/2011 02:52 PM 731,106 FileMap.ModifiedIndex-f-50-Index.db
03/29/2011 02:52 PM 4,264 FileMap.ModifiedIndex-f-50-Statistics.db
03/30/2011 05:37 PM 2,065 FileMap.ModifiedIndex-f-51-Data.db
03/30/2011 05:37 PM 64 FileMap.ModifiedIndex-f-51-Filter.db
03/30/2011 05:37 PM 450 FileMap.ModifiedIndex-f-51-Index.db
03/30/2011 05:37 PM 4,264 FileMap.ModifiedIndex-f-51-Statistics.db
04/04/2011 04:07 PM 13,835 FileMap.ModifiedIndex-f-52-Data.db
04/04/2011 04:07 PM 328 FileMap.ModifiedIndex-f-52-Filter.db
04/04/2011 04:07 PM 3,006 FileMap.ModifiedIndex-f-52-Index.db
04/04/2011 04:07 PM 4,264 FileMap.ModifiedIndex-f-52-Statistics.db
04/04/2011 04:07 PM 962,874 FileMap.UserIndex-f-53-Data.db
04/04/2011 04:07 PM 1,936 FileMap.UserIndex-f-53-Filter.db
04/04/2011 04:07 PM 420 FileMap.UserIndex-f-53-Index.db
04/04/2011 04:07 PM 4,264 FileMap.UserIndex-f-53-Statistics.db
In the search, we are using IndexClause as:
ByteBuffer field_author = ByteBuffer.wrap(new byte[]{'a'});
ByteBuffer author_1 = IntegerSerializer.get().toByteBuffer(1);
ByteBuffer file_type = ByteBuffer.wrap(new byte[]{'t'});
ByteBuffer filetype_3 = ByteBuffer.wrap(new byte[]{3}); //file
type 3
IndexClause indexClause = new IndexClause();
indexClause.setCount(3000);
ArrayList<IndexExpression> expressions = new ArrayList();
expressions.add(new IndexExpression(field_author,
IndexOperator.EQ, author_1)); //user ID = 1
expressions.add(new IndexExpression(file_type,
IndexOperator.EQ, filetype_3)); //file type = 3
indexClause.setExpressions(expressions);
indexClause.setStart_key(new byte[]{});
}}
In the search, it scans all the indices from "FileMap.UserIndex", within which
there seems having a key (index) which is not found in the table "FileMap"; and
I roughly get that it breaks at data retrieval with "FileMap-f-53-Data", when
the position for the key is not found / available in "FileMap-f-53-Data".
> getColumnFamily() return null, which is not checked in ColumnFamilyStore.java
> scan() method, causing Timeout Exception in query
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2401
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.4
> Environment: Hector 0.7.0-28, Cassandra 0.7.4, Windows 7, Eclipse
> Reporter: Tey Kar Shiang
>
> ColumnFamilyStore.java, line near 1680, "ColumnFamily data =
> getColumnFamily(new QueryFilter(dk, path, firstFilter))", the data is
> returned null, causing NULL exception in "satisfies(data, clause, primary)"
> which is not captured. The callback got timeout and return a Timeout
> exception to Hector.
> The data is empty, as I traced, I have the the columns Count as 0 in
> removeDeletedCF(), which return the null there. (I am new and trying to
> understand the logics around still). Instead of crash to NULL, could we
> bypass the data?
> About my test:
> A stress-test program to add, modify and delete data to keyspace. I have 30
> threads simulate concurrent users to perform the actions above, and do a
> query to all rows periodically. I have Column Family with rows (as File) and
> columns as index (e.g. userID, fileType).
> No issue on the first day of test, and stopped for 3 days. I restart the test
> on 4th day, 1 of the users failed to query the files (timeout exception
> received). Most of the users are still okay with the query.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
