What version of Solr are you using, and how are you indexing? DIH? SolrJ? I'm guessing you're using Tika, but how?
Best Erick On Tue, Aug 30, 2011 at 4:55 AM, Marc Jacobs <jacob...@gmail.com> wrote: > Hi all, > > Currently I'm testing Solr's indexing performance, but unfortunately I'm > running into memory problems. > It looks like Solr is not closing the filestream after an exception, but I'm > not really sure. > > The current system I'm using has 150GB of memory and while I'm indexing the > memoryconsumption is growing and growing (eventually more then 50GB). > In the attached graph I indexed about 70k of office-documents (pdf,doc,xls > etc) and between 1 and 2 percent throws an exception. > The commits are after 64MB, 60 seconds or after a job (there are 6 evenly > divided jobs). > > After indexing the memoryconsumption isn't dropping. Even after an optimize > command it's still there. > What am I doing wrong? I can't imagine I'm the only one with this problem. > Thanks in advance! > > Kind regards, > > Marc >
