John Call created SOLR-9579:
-------------------------------
Summary: Reuse lucene FieldType in createField flow during
ingestion
Key: SOLR-9579
URL: https://issues.apache.org/jira/browse/SOLR-9579
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: Schema and Analysis
Affects Versions: 6.x, master (7.0)
Environment: This has been primarily tested on Windows 8 and Windows
Server 2012 R2
Reporter: John Call
Priority: Minor
Fix For: 6.x, master (7.0)
During ingestion createField in FieldType is being called for each field on
each document. For the subclasses of FieldType without their own implementation
of createField the lucene version of FieldType is created to be stored along
with the value. However the lucene FieldType object is identical when created
from the same SchemaField. In testing ingestion of one million rows with 22
field each we were creating 22 million lucene FieldType objects when only 22
are needed. Solr should lazily initialize a lucene FieldType for each
SchemaField and reuse them for future ingestion. Not only does this relieve
memory usage but also relieves significant pressure on the gc.
There are also subclasses of Solr FieldType which create separate Lucene
FieldType for stored fields instead of reusing the static in StoredField.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
