Shawn Heisey created SOLR-10014:
-----------------------------------
Summary: Log a warning when the number of fields in a core exceeds
a configurable value
Key: SOLR-10014
URL: https://issues.apache.org/jira/browse/SOLR-10014
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 4.10.4
Reporter: Shawn Heisey
Priority: Minor
When the number of fields in an index gets extremely large, major performance
problems can occur. If the number of fields in a core exceeds a configurable
number, with a default somewhere around 10000, a warning should be logged when
the SolrCore is first created. A decision needs to be made about whether to
repeat the warning on core reload ... my instinct is that it should NOT be
repeated, but I can see where a repeat might have some value. Logging on
reloads as well as startup would likely be easier.
This was discovered by a Solr user who had a 420MB index with 650K documents,
but their applications were abusing dynamic fields to the point where they had
about 2 million unique fields in the index. The small size of the index
*should* have resulted in extremely fast commit times, but commits were taking
about 10 seconds because of what Lucene had to do to handle all those fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
