And if it is on Solr side, please increase the heap size on Solr side https://cwiki.apache.org/confluence/display/solr/JVM+Settings
On Fri, Feb 19, 2016 at 8:42 AM, Susheel Kumar <susheel2...@gmail.com> wrote: > When you run your SolrJ Client Indexing program, can you increase heap > size similar below. I guess it may be on your client side you are running > int OOM... or please share the exact error if below doesn't work/is the > issue. > > java -Xmx4096m .... > > > Thanks, > > Susheel > > On Fri, Feb 19, 2016 at 6:25 AM, Clemens Wyss DEV <clemens...@mysign.ch> > wrote: > >> Guessing on ;) : >> must I commit after every "batch", in order to force a flushing of >> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream et al? >> >> OTH it is propagated to NOT "commit" from a (SolrJ) client >> >> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >> 'Be very careful committing from the client! In fact, don’t do it' >> >> I would not want to commit "just to flush a client side buffer" ... >> >> -----Ursprüngliche Nachricht----- >> Von: Clemens Wyss DEV [mailto:clemens...@mysign.ch] >> Gesendet: Freitag, 19. Februar 2016 11:09 >> An: solr-user@lucene.apache.org >> Betreff: AW: OutOfMemory when batchupdating from SolrJ >> >> The char[] which occupies 180MB has the following "path to root" >> >> char[87690841] @ 0x7940ba658 <add><doc boost="1.0"><field >> name="_my_id">shopproducts#<CUT>... >> |- <Java Local> java.lang.Thread @ 0x7321d9b80 SolrUtil executorService >> |for core 'fust-1-fr_CH_1' -3-thread-1 Thread >> |- value java.lang.String @ 0x79e804110 <add><doc boost="1.0"><field >> name="_my_id">shopproducts#<CUT>... >> | '- str org.apache.solr.common.util.ContentStreamBase$StringStream @ >> 0x77fd84680 >> | |- <Java Local> java.lang.Thread @ 0x7321d9b80 SolrUtil >> executorService for core 'fust-1-fr_CH_1' -3-thread-1 >> | |- contentStream >> org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream @ >> 0x77fd846a0 >> | | |- <Java Local> java.lang.Thread @ 0x7321d9b80 SolrUtil >> executorService for core 'fust-1-fr_CH_1' -3-thread-1 Thread >> | | |- [0] org.apache.solr.common.util.ContentStream[1] @ 0x79e802fb8 >> | | | '- <Java Local> java.lang.Thread @ 0x7321d9b80 SolrUtil >> |executorService for core 'fust-1-fr_CH_1' -3-thread-1 Thread >> >> And there is another byte[] with 260MB. >> >> The logic is somewhat this: >> >> SolrClient solrClient = new HttpSolrClient( coreUrl ); while ( got more >> elements to index ) { >> batch = create 100 SolrInputDocuments >> solrClient.add( batch ) >> } >> >> >> -----Ursprüngliche Nachricht----- >> Von: Clemens Wyss DEV [mailto:clemens...@mysign.ch] >> Gesendet: Freitag, 19. Februar 2016 09:07 >> An: solr-user@lucene.apache.org >> Betreff: OutOfMemory when batchupdating from SolrJ >> >> Environment: Solr 5.4.1 >> >> I am facing OOMs when batchupdating SolrJ. I am seeing approx 30'000(!) >> SolrInputDocument instances, although my batchsize is 100. I.e. I call >> solrClient.add( documents ) for every 100 documents only. So I'd expect to >> see at most 100 SolrInputDocument's in memory at any moment UNLESS >> a) solrClient.add is "asynchronous" in its nature. Then QueryResponse >> would be an async-result? >> or >> b) SolrJ is spooling the documents in client-side >> >> What might be going wrong? >> >> Thx for your advices >> Clemens >> >> >
