[
https://issues.apache.org/jira/browse/SOLR-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234401#comment-16234401
]
Nawab Zada Asad iqbal commented on SOLR-11504:
----------------------------------------------
[~dsmiley]
You have duplicated it with "SOLR-3585" . Isn't that JIRA very broad scoped?
The scope in the current ticket (11504) is to restrict the requests from Solr
to Lucene's `IndexWriter`. My initial thoughts are: IndexWriter.getDocument(s)
and updateDocument(s) is mostly used from `DirectUpdateHandler2` (It is also
used in `FileBasedSpellChecker.java` : which seems to be a non-routine
scenario). For the purpose of fixing SOLR-11504, it seems enough to use a
counting semaphore (or any similar structure) to control the flow of indexing
requests from `DirectUpdateHandler2` to `IndexWriter`.
What do you think?
> Provide a config to restrict number of indexing threads
> --------------------------------------------------------
>
> Key: SOLR-11504
> URL: https://issues.apache.org/jira/browse/SOLR-11504
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 5.3, 6.0, 7.0
> Reporter: Nawab Zada Asad iqbal
> Priority: Major
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> For heavy indexing load (through REST api), Solr does not have any way to
> restrict number of threads. There used to be a config in lucene to restrict
> number of threads but that has been removed since
> https://issues.apache.org/jira/browse/LUCENE-6659 .
> For example, in my bulk indexing scenario, within few minutes, my solr server
> had created 300 parallel threads each writing its own segment. The result was
> tons of small segments getting flushed to disk (as total RAM limit was
> reached quickly by sum of all segments), and solr has to spend time later to
> merge them into reasonable sizes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
