[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577420#comment-13577420
]
Sylvain Lebresne commented on CASSANDRA-5222:
---------------------------------------------
bq. I noticed that Validator.add already asserts that the row is in the range
being validated, so we must be doing a range check somewhere.
The validation is intrinsically limited to a range, so
ValidationCompactionIterable make sure we only scan said range through
{noformat}
cfs.getCompactionStrategy().getScanners(sstables, range)
{noformat}
However, looking at it, it's not very efficient because currently we end up
creating an SSTableBoundedScanner object even if the range is not covered at
all by the sstable. And while said SSTableBoundedScanner will be correct in
that it will be exhausted right away, the way the code works, we still open the
data file and so we still allocate the reader buffer(s). So, probably a
separate issue but we should optimize that nonetheless. And while looking at
that, I realized that LeveledScanner poorly compute the getLengthInBytes info
as it assumes we cover the whole ring (we will still end up at 100% because
getCurrentPosition() will also return a value assuming the whole ring, but it's
still a poor indicator of the actual work we do/have to do).
> OOM Exception during repair session with LeveledCompactionStrategy
> ------------------------------------------------------------------
>
> Key: CASSANDRA-5222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Environment: 3Gb Heap(12Gb per node RAM)
> 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
> =100Mb
> Reporter: Ivan Sobolev
> Assignee: Jonathan Ellis
> Fix For: 1.1.11
>
> Attachments: 5222.txt, chunks.json, sstablescanner.png
>
>
> 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
