Our use case is the following: an external process generates 70 images, each around ~700k in size. These are uploaded as sub-nodes under a master node that encapsulates the run. There are are also some sister nodes that contain a modest amount of metadata about each image and the run that generated it. In general most of the writing consists of a client POSTing these images into the repository via Sling; there are then some event handlers and tasks that look at the data that arrived. The only subsequent writes at present are some properties that are set after these images are examined and replicated into another system. So, I don't expect much at all in the way of concurrent read/write; it's mainly write a bunch and then read it back later.
By heavy pressure what I mean is that we have a test lab running continuously against this system. It's a lot more traffic than can be expected in the real world, but it is good for shaking out problems. What concerns me is that according to the documentation an entire Lucene index is kept in memory. Right now we don’t do any pruning - our repository only grows larger. This implies to me that the index will only grow as well and we will ultimately run out of memory no matter how big the heap is. Hopefully I'm wrong about that. At the moment we have no JVM flags set. The SearchIndex configuration is also default (by default I mean what came with Sling), although I am looking at turning off supportHighlighting and putting a small value for resultFetchSize, say 100. -----Original Message----- From: Ben Frisoni [mailto:[email protected]] Sent: Monday, November 23, 2015 11:55 AM To: [email protected] Subject: Re: Memory usage A little bit of description on the term heavy pressure might help? Does this involve concurrent read operations/ write operations or both? Also some other things that effect performance: 1. What jvm parameters are set? 2. Do you have any custom index configurations set? 3. What does you repostiory.xml look like? This background info might help with answering your question. On Mon, Nov 23, 2015 at 8:13 AM, Roll, Kevin <[email protected]> wrote: > We have started to encounter OutOfMemory errors on Jackrabbit under heavy > pressure (it's worth noting that we are using the full Sling stack). I've > discovered that Lucene keeps a full index of the repository in memory, and > this terrifies me because we are already having problems just in a test > scenario and the repository will only grow. Unfortunately we are forced to > run this system on older 32-bit hardware in the field that does not have > any room to expand memory-wise. Are there any options I can tweak to reduce > the memory footprint? Any other things I can disable that will cut down on > memory usage? Is Oak better in this regard? Thanks! > >
