I think you should try isolating the problem. It may turn out that the problem isn't really to do with Solr, but file uploading. I'm no expert, but that's what I'd try out in such situation.
Cheers, Timothy Wonil Lee http://timundergod.blogspot.com/ http://www.google.com/reader/shared/16849249410805339619 -----Original Message----- From: David Thibault [mailto:[EMAIL PROTECTED] Sent: Thursday, 17 January 2008 8:30 AM To: solr-user@lucene.apache.org Subject: Re: Indexing very large files. OK, I have now bumped my tomcat JVM up to 1024MB min and 1500MB max. For some reason Walter's suggestion helped me get past the 8MB file upload to Solr but it's still choking on a 32MB file. Is there a way to set per-webapp JVM settings in tomcat, or is the overall tomcat JVM sufficient to set? I can't see anything in the tomcat manager to suggest that there are smaller memory limitations for solr or any other webapp (all the demo webapps that tomcat comes with are still there right now). Here's the trace I get when I try to upload the 32MB file: java.lang.OutOfMemoryError: Java heap space at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java :95) at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java :61) at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java :336) at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(StreamEncoder.java :395) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:136) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:191) at com.itstrategypartners.sents.solrUpload.SimplePostTool.pipe( SimplePostTool.java:167) at com.itstrategypartners.sents.solrUpload.SimplePostTool.postData( SimplePostTool.java:125) at com.itstrategypartners.sents.solrUpload.SimplePostTool.postFile( SimplePostTool.java:87) at com.itstrategypartners.sents.solrUpload.Uploader.uploadFile( Uploader.java:97) at com.itstrategypartners.sents.solrUpload.UploaderTest.uploadFile( UploaderTest.java:95) Any more thoughts on possible causes? Best, Dave On 1/16/08, David Thibault <[EMAIL PROTECTED]> wrote: > > Walter and all, > > I had been bumping up the heap for my Java app (running outside of Tomcat) > but I hadn't yet tried bumping up my Tomcat heap. That seems to have helped > me upload the 8MB file, but it's crashing while uploading a 32MB file now. I > Just bumped tomcat to 1024MB of heap, so I'm not sure what the problem is > now. I suspect Walter was on to something, since it sort of fixed my > problem. I will keep troubleshooting the Tomcat memory and go from there.. > > > Best, > Dave > > On 1/16/08, Walter Underwood < [EMAIL PROTECTED]> wrote: > > > > This error means that the JVM has run out of heap space. Increase the > > heap space. That is an option on the "java" command. I set my heap to > > 200 Meg and do it this way with Tomcat 6: > > > > JAVA_OPTS="-Xmx600M" tomcat/bin/startup.sh > > > > wunder > > > > On 1/16/08 8:33 AM, "David Thibault" < [EMAIL PROTECTED]> > > wrote: > > > > > java.lang.OutOfMemoryError: Java heap space > > > > > > !DSPAM:478e7768324633671820667!
