[
https://issues.apache.org/jira/browse/SOLR-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786161#action_12786161
]
Yonik Seeley commented on SOLR-1623:
------------------------------------
This is most likely due to interning of field names. If you really need that
many field names, the only option right now is to increase the size of the perm
gen.
> Solr hangs (often throwing java.lang.OutOfMemoryError: PermGen space) when
> indexing many different field names
> --------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-1623
> URL: https://issues.apache.org/jira/browse/SOLR-1623
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 1.3, 1.4
> Environment: Tomcat Version JVM Version
> JVM Vendor OS Name OS Version OS Architecture
> Apache Tomcat/6.0 snapshot 1.6.0_13-b03 Sun Microsystems Inc. Linux
> 2.6.18-164.el5 amd64
> and/or
> Tomcat Version JVM Version JVM Vendor
> OS Name OS Version OS Architecture
> Apache Tomcat/6.0.18 1.6.0_12-b04 Sun Microsystems Inc. Windows
> 2003 5.2 amd64
> Reporter: Laurent Chavet
> Priority: Critical
>
> With the following fields in schema.xml:
> <fields>
> <field name="id" type="sint" indexed="true" stored="true" required="true"
> />
> <dynamicField name="weight_*" type="sint" indexed="true"
> stored="true"/>
> </fields>
> Run the following code:
> import java.util.ArrayList;
> import java.util.List;
> import org.apache.solr.client.solrj.SolrServer;
> import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
> import org.apache.solr.common.SolrInputDocument;
> public static void main(String[] args) throws Exception {
> SolrServer server;
> try {
> server = new CommonsHttpSolrServer(args[0]);
> } catch (Exception e) {
> System.err.println("can't creater server using: " + args[0] + "
> " + e.getMessage());
> throw e;
> }
> for (int i = 0; i < 1000; i++) {
> List<SolrInputDocument> batchedDocs = new
> ArrayList<SolrInputDocument>();
> for (int j = 0; j < 1000; j++) {
> SolrInputDocument doc = new SolrInputDocument();
> doc.addField("id", i * 1000 + j);
> // hangs after 30 to 50 batches
>
> doc.addField("weight_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
> + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j);
> // hangs after about 200 batches
> //doc.addField("weight_" + Integer.toString(i) + "_" +
> Integer.toString(j), i * 1000 + j);
> batchedDocs.add(doc);
> }
> try {
> server.add(batchedDocs, true);
> System.err.println("Done with batch=" + i);
> // server.commit(); //doesn't change anything
> } catch (Exception e) {
> System.err.println("batchId=" + i + " bad batch: " +
> e.getMessage());
> throw e;
> }
> }
> }
> And soon the client (sometime throws) and solr will freeze. sometime you can
> see: java.lang.OutOfMemoryError: PermGen space in the server logs
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
