[
https://issues.apache.org/jira/browse/CASSANDRA-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193183#comment-13193183
]
Vijay commented on CASSANDRA-3762:
----------------------------------
Bigger test with much spread out data showed some mixed results.... Tested on
M24XL nodes in AWS, 8M Keys inserted via Stress tool 3 times to get
distribution of the keys into the SST's
Trunk shows:
DEBUG [SSTableBatchOpen:1] 2012-01-25 09:48:28,098 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/system/LocationInfo/system-LocationInfo-hc-17: 16
ms.
DEBUG [SSTableBatchOpen:6] 2012-01-25 09:48:47,699 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-637:
4289 ms.
DEBUG [SSTableBatchOpen:3] 2012-01-25 09:48:51,541 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-607:
8136 ms.
DEBUG [SSTableBatchOpen:5] 2012-01-25 09:48:52,910 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-429:
9504 ms.
DEBUG [SSTableBatchOpen:1] 2012-01-25 09:49:07,725 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-611:
24321 ms.
DEBUG [SSTableBatchOpen:7] 2012-01-25 09:49:29,760 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-534:
46348 ms.
DEBUG [SSTableBatchOpen:8] 2012-01-25 09:49:34,972 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-462:
51560 ms.
DEBUG [SSTableBatchOpen:2] 2012-01-25 09:49:35,893 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-636:
52488 ms.
DEBUG [SSTableBatchOpen:4] 2012-01-25 09:49:45,671 SSTableReader.java (line
193) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-635:
62258 ms
JMX key cache size:
01/25/2012 09:50:18 +0000 org.archive.jmx.Client KeyCacheSize: 104857584
After this patch:
DEBUG [SSTableBatchOpen:6] 2012-01-25 18:22:33,343 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-637:
5731 ms.
DEBUG [SSTableBatchOpen:6] 2012-01-25 18:22:33,344 SSTableReader.java (line
198) key cache contains 2/2184533 keys
DEBUG [SSTableBatchOpen:3] 2012-01-25 18:22:34,291 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-607:
6685 ms.
DEBUG [SSTableBatchOpen:3] 2012-01-25 18:22:34,291 SSTableReader.java (line
198) key cache contains 7619/2184533 keys
DEBUG [SSTableBatchOpen:1] 2012-01-25 18:22:37,144 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-611:
9541 ms.
DEBUG [SSTableBatchOpen:1] 2012-01-25 18:22:37,145 SSTableReader.java (line
198) key cache contains 45172/2184533 keys
DEBUG [SSTableBatchOpen:5] 2012-01-25 18:22:37,275 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-429:
9664 ms.
DEBUG [SSTableBatchOpen:5] 2012-01-25 18:22:37,276 SSTableReader.java (line
198) key cache contains 48914/2184533 keys
DEBUG [SSTableBatchOpen:2] 2012-01-25 18:22:42,224 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-636:
14620 ms.
DEBUG [SSTableBatchOpen:2] 2012-01-25 18:22:42,224 SSTableReader.java (line
198) key cache contains 170841/2184533 keys
DEBUG [SSTableBatchOpen:4] 2012-01-25 18:22:45,053 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-635:
17442 ms.
DEBUG [SSTableBatchOpen:4] 2012-01-25 18:22:45,053 SSTableReader.java (line
198) key cache contains 241841/2184533 keys
DEBUG [SSTableBatchOpen:8] 2012-01-25 18:23:01,720 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-462:
34107 ms.
DEBUG [SSTableBatchOpen:8] 2012-01-25 18:23:01,720 SSTableReader.java (line
198) key cache contains 689699/2184533 keys
DEBUG [SSTableBatchOpen:7] 2012-01-25 18:24:19,975 SSTableReader.java (line
195) INDEX LOAD TIME for
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-534:
112363 ms.
DEBUG [SSTableBatchOpen:7] 2012-01-25 18:24:19,975 SSTableReader.java (line
198) key cache contains 2184533/2184533 keys
Sizes of the index:
1068
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-637-Index.db
7852
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-429-Index.db
20044
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-607-Index.db
84600
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-611-Index.db
188560
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-534-Index.db
212204
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-636-Index.db
217292
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-462-Index.db
315988
/mnt/data/cassandra070/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-635-Index.db
If we want to see the optimal solution for all the use cases i think we have to
go for the alternative where we can save the Keycache position to the disk and
read it back and what ever is missing let it fault fill. Agree?
> AutoSaving KeyCache and System load time improvements.
> ------------------------------------------------------
>
> Key: CASSANDRA-3762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3762
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.2
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.2
>
> Attachments: 0001-SavedKeyCache-load-time-improvements.patch
>
>
> CASSANDRA-2392 saves the index summary to the disk... but when we have saved
> cache we will still scan through the index to get the data out.
> We might be able to separate this from SSTR.load and let it load the index
> summary, once all the SST's are loaded we might be able to check the
> bloomfilter and do a random IO on fewer Index's to populate the KeyCache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
