Hi Shawn, I've tried to set 4GB heap for Solr and the OOM exception rellay get reduce and also performance gained.
Floyd 2014-06-16 0:00 GMT+08:00 Shawn Heisey <s...@elyograg.org>: > On 6/15/2014 2:54 AM, Floyd Wu wrote: > > Thank you Alex. > > I'm doing commit every 100 fiels. > > Maybe there is a better way to do this job, something like DIH(possible?) > > Sometimes i have much bigger xml file (2MB) and post to SOLR(jetty > enabled) > > may encounter slow or exceed limitation. > > If you are getting OOM exceptions on your Solr server, then you need to > increase your Java heap size. I have never seen the "pending too many" > error that you mentioned at the beginning of the thread, and Google > didn't turn up anything useful, so I don't know what needs to be done > for that. If you can post the entire exception stacktrace for this > error, perhaps we can figure it out. We would also need the exact Solr > version. > > Solr has a default 2MB limit on POST requests. This can be increased > with the formdataUploadLimitInKB parameter on the requestDispatcher tag > in solrconfig.xml -- assuming that you're running 4.1 or later. > Previous versions required changing the request size in the servlet > container config, but there was a bug in the example Jetty included in > 4.0.0 that made it impossible to change the size. > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=397130 > https://issues.apache.org/jira/browse/SOLR-4223 > > Regarding the advice to disable the updateLog: I never like to do this, > but if you are sending a very large number of updates in a single > request, it might be advisable until indexing is complete, so that Solr > restart times are not excessive. > > Thanks, > Shawn > >
