[
https://issues.apache.org/jira/browse/COUCHDB-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910328#action_12910328
]
Adam Kocoloski commented on COUCHDB-888:
----------------------------------------
Hi Damien, I guess the implications of limiting the number of edit branches
would be pretty serious, right? When we stem a branch we run the risk of
introducing spurious conflicts, but if we limited the number of branches it
seems like we'd be forced to throw away data.
I agree that if the tree is memory-resident there will always be a point at
which it will crash the VM, but if we can architect things so that point only
occurs when the tree itself requires gigabytes of memory we'd be in pretty good
shape. I have the sense that the offending tree is really only ~10 MB, but
that the traversal does some N^2 explosion. I could be wrong.
Disk-resident would be really cool if the most common interactions still only
require one lookup.
> out of memory crash when compacting document with lots of edit branches
> -----------------------------------------------------------------------
>
> Key: COUCHDB-888
> URL: https://issues.apache.org/jira/browse/COUCHDB-888
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Adam Kocoloski
> Attachments: key_tree_backtrace.txt.gz
>
>
> I have a database which will crash CouchDB if I try to compact it. It causes
> beam.smp to use all the memory on the server. I caught it in the act one
> time and sorted the Erlang processes by memory usage. The process spawned to
> do the compaction turned out to be the culprit. I took a backtrace of the
> process and found that it was mapping a very large revision tree. I have
> reason to believe that the document has a large number (~1000s) of edit
> conflicts.
> I think part of the problem may be that the recursion in
> couch_key_tree:map_simple requires each stack space for every iteration. I'm
> not sure if it's possible to rewrite the algorithm in a more memory-friendly
> way given the current tree structure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
