[
https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977801#action_12977801
]
T Jake Luciani commented on CASSANDRA-1902:
-------------------------------------------
Well the good news is the mincore() stuff works via JNA!
I'm re-considering what todo with this information. It's most efficient to
keep around contiguous chunks of pages so the plan might be to find the most
densely populated ranges of pages in the sstable then get the range of rows
this covered. Then pass to the SSTableWriter this list which will subsequently
mark the new data written for these pages as POSIX_FADV_WILLNEED. The ordering
of the keys should be close.
I think in the average case this will get the most active data cached. It may
keep too much data in the page cache though.
Any thoughts on this?
> Migrate cached pages during compaction
> ---------------------------------------
>
> Key: CASSANDRA-1902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1902
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7.1
> Reporter: T Jake Luciani
> Assignee: T Jake Luciani
> Fix For: 0.7.1
>
> Original Estimate: 32h
> Remaining Estimate: 32h
>
> Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a
> pre-compacted CF during the compaction process.
> First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that
> uses the posix mincore() function to detect the offsets of pages for this
> file currently in page cache.
> Then add getActiveKeys() which uses underlying pagesInPageCache() to get the
> keys actually in the page cache.
> use getActiveKeys() to detect which SSTables being compacted are in the os
> cache and make sure the subsequent pages in the new compacted SSTable are
> kept in the page cache for these keys. This will minimize the impact of
> compacting a "hot" SSTable.
> A simpler yet similar approach is described here:
> http://insights.oetiker.ch/linux/fadvise/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
