Hm... very interested in a solution too as we see this happening - on imports - on exports - on deletion of massive nodes.
For starters if you don't have all article in a single-level structure (i.e. if you have folders and subfolders) you can try to open the XML in a good text editor and split it up into several chunks which can then be partially imported. -will On 13.12.2010, at 10:44, Ernst Bunders wrote: > Hello > > We are in the final stages of (re)creating one of our largest sites in > magnolia (http://geschiedenis.vpro.nl). This site has a massive archive of > articles and dossiers which we all diligently imported in the new site. > > So, the final step is to export the data from our test environment and re > import it on the acceptance server. This is where trouble hits. I have a file > website.geschiedenis.xml that is quite large. it's size is 292Mb, and > cat website.geschiedenis.xml | grep "<sv:node" | wc -l > yields: 289648. Quite a lot of nodes. > > The problem is that we can not import this file anymore. With max heap space > settings of 2Gb we still get OutOfMemory exceptions. > > Apparently this whole xml tree is being constructed in memory, in a > not-so-efficient way (lots of overhead if you can't fit 292 Mb of data in a > 2Gb tree) > > We do not have a lot of knowledge about this subject yet. We don't know if > and how Jackrabbit can be tuned to deal with different scenario's. So any > help would be appreciated. This is clearly a problem we need to get out of > the way, also because it impedes our possibilities for backup and restore. > > For now I am going to try to cut the file up into different smaller ones, see > how that will go. > > regards, > > Ernst > > -- > Ernst Bunders > Ontwikkelaar VPRO ---------------------------------------------------------------- For list details see http://www.magnolia-cms.com/home/community/mailing-lists.html To unsubscribe, E-mail to: <[email protected]> ----------------------------------------------------------------
