[
https://issues.apache.org/jira/browse/CASSANDRA-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033432#comment-13033432
]
Jonathan Ellis commented on CASSANDRA-1608:
-------------------------------------------
bq. client supplied timestamps mean that you can't know that newer files
supercede older ones
Right, but I see that as an orthogonal concern. (CASSANDRA-2498 will provide
similar benefits to reads as being able to sort by timestamp, but again, that's
basically independent of compaction strategy.)
2) the CF data model means that data for a given key in multiple sstables may
need to be merged
Also right. I don't think changing that (or changing that you may have to merge
data from multiple sstables on reads) should be a goal for us.
I guess I wasn't clear; I'm not proposing "let's try to make Cassandra fit in
the leveldb design."
Compaction is about "how do we avoid rewriting the same data over and over,
while minimizing the space penalty from not doing overwrites" and the leveldb
approach almost certainly does better on both of those metrics than our current
approach, specfically because of the non-overlapping-sstables-within-a-Level
approach. That's the interesting part to me.
> Redesigned Compaction
> ---------------------
>
> Key: CASSANDRA-1608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1608
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Chris Goffinet
>
> After seeing the I/O issues in CASSANDRA-1470, I've been doing some more
> thinking on this subject that I wanted to lay out.
> I propose we redo the concept of how compaction works in Cassandra. At the
> moment, compaction is kicked off based on a write access pattern, not read
> access pattern. In most cases, you want the opposite. You want to be able to
> track how well each SSTable is performing in the system. If we were to keep
> statistics in-memory of each SSTable, prioritize them based on most accessed,
> and bloom filter hit/miss ratios, we could intelligently group sstables that
> are being read most often and schedule them for compaction. We could also
> schedule lower priority maintenance on SSTable's not often accessed.
> I also propose we limit the size of each SSTable to a fix sized, that gives
> us the ability to better utilize our bloom filters in a predictable manner.
> At the moment after a certain size, the bloom filters become less reliable.
> This would also allow us to group data most accessed. Currently the size of
> an SSTable can grow to a point where large portions of the data might not
> actually be accessed as often.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
