Solr hangs (often throwing java.lang.OutOfMemoryError: PermGen space) when
indexing many different field names
--------------------------------------------------------------------------------------------------------------
Key: SOLR-1623
URL: https://issues.apache.org/jira/browse/SOLR-1623
Project: Solr
Issue Type: Bug
Components: update
Affects Versions: 1.4, 1.3
Environment: Tomcat Version JVM Version
JVM Vendor OS Name OS Version OS Architecture
Apache Tomcat/6.0 snapshot 1.6.0_13-b03 Sun Microsystems Inc. Linux
2.6.18-164.el5 amd64
and/or
Tomcat Version JVM Version JVM Vendor
OS Name OS Version OS Architecture
Apache Tomcat/6.0.18 1.6.0_12-b04 Sun Microsystems Inc. Windows
2003 5.2 amd64
Reporter: Laurent Chavet
Priority: Critical
With the following fields in schema.xml:
<fields>
<field name="id" type="sint" indexed="true" stored="true" required="true" />
<dynamicField name="weight_*" type="sint" indexed="true"
stored="true"/>
</fields>
Run the following code:
import java.util.ArrayList;
import java.util.List;
import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
import org.apache.solr.common.SolrInputDocument;
public static void main(String[] args) throws Exception {
SolrServer server;
try {
server = new CommonsHttpSolrServer(args[0]);
} catch (Exception e) {
System.err.println("can't creater server using: " + args[0] + " "
+ e.getMessage());
throw e;
}
for (int i = 0; i < 1000; i++) {
List<SolrInputDocument> batchedDocs = new
ArrayList<SolrInputDocument>();
for (int j = 0; j < 1000; j++) {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", i * 1000 + j);
// hangs after 30 to 50 batches
doc.addField("weight_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
+ Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j);
// hangs after about 200 batches
//doc.addField("weight_" + Integer.toString(i) + "_" +
Integer.toString(j), i * 1000 + j);
batchedDocs.add(doc);
}
try {
server.add(batchedDocs, true);
System.err.println("Done with batch=" + i);
// server.commit(); //doesn't change anything
} catch (Exception e) {
System.err.println("batchId=" + i + " bad batch: " +
e.getMessage());
throw e;
}
}
}
And soon the client (sometime throws) and solr will freeze. sometime you can
see: java.lang.OutOfMemoryError: PermGen space in the server logs
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
