[
https://issues.apache.org/jira/browse/CASSANDRA-5555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686768#comment-13686768
]
Jonathan Ellis commented on CASSANDRA-5555:
-------------------------------------------
Ah, right. Related to CASSANDRA-5542.
I think we can simplify a bit though. Leave out the option, and have
sstableloader load if present, but just discard it after load instead of
keeping it memory resident.
May also be simpler to go through the "generate summary if it doesn't exist"
path than add separate "count keys if we don't have a summary" code.
Personally I'd prefer to just make summary required, but we probably can't do
that this late in 1.2.
Is this messy enough that we should just revert and do this in trunk? On the
bright side we could make summary required. :)
> Allow sstableloader to handle a larger number of files
> ------------------------------------------------------
>
> Key: CASSANDRA-5555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5555
> Project: Cassandra
> Issue Type: Improvement
> Components: Core, Tools
> Reporter: Tyler Hobbs
> Assignee: Jonathan Ellis
> Fix For: 1.2.6
>
> Attachments: 5555-01.txt, 5555-02.txt, CASSANDRA-5555.txt,
> CASSANDRA-5555.txt
>
>
> With the default heap size, sstableloader will OOM when there are roughly 25k
> files in the directory to load. It's easy to reach this number of files in a
> single LCS column family.
> By avoiding creating all SSTableReaders up front in SSTableLoader, we should
> be able to increase the number of files that sstableloader can handle
> considerably.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
