DataImportHandler might be a better way to import very large XML files if it can be loaded from Solr-local file system.
Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Tue, Apr 23, 2013 at 9:46 AM, Shawn Heisey <s...@elyograg.org> wrote: > On 4/23/2013 6:02 AM, Sharmila Thapa wrote: >> What is the maximum size limit of the XML document file that is allowed to >> import into solr to index from java -Durl. As I am testing to import XMLfile >> of 5 GB and it throws an error like >> SimplePostTool: WARNING: Solr returned an error #400 Bad Request >> SimplePostTool: WARNING: IOException while reading response: >> java.io.IOException: Server returned HTTP response code: 400 for URL: >> http://10.0.1.140:8080/solr/solr1/update > > Unless the simple post tool is capable of breaking the input XML into > many pieces, you'll run into the POST size limit of your servlet > container. I don't know if it has this capability, but I would be > somewhat surprised if it did. > > Solr is packaged so the example uses jetty (start.jar), but you may be > running under tomcat or one of a few other choices. The history of the > POST limit in Solr is a little complex. > > The example jetty config in Solr 3.x (and possibly earlier) used a 1MiB > POST buffer. You could change that value with no problem. If you used > another container, you could change it using that container's > configuration method. > > When 4.0 was released, jetty 8.x had a bug and the 1MiB configuration in > the example wasn't working, so the limit became 200KB, jetty's default. > Just like earlier versions, if you were using another container, you > could change the limit using that container's configuration. > > The bug in jetty has now been fixed. > https://bugs.eclipse.org/bugs/show_bug.cgi?id=397130 > > Solr 4.1 changed things, with SOLR-4265. Now Solr controls the max POST > size itself, defaulting formdataUploadLimitInKB in solrconfig.xml to 2048. > > https://issues.apache.org/jira/browse/SOLR-4265 > > Thanks, > Shawn >
