[
https://issues.apache.org/jira/browse/SOLR-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245257#comment-14245257
]
Varun Thacker commented on SOLR-6845:
-------------------------------------
Excerpt from the logs -
{code}
16160 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.handler.component.SpellCheckComponent – Initializing spell
checkers
16161 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.spelling.DirectSolrSpellChecker – init:
{name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
16162 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.handler.component.SpellCheckComponent – No queryConverter
defined, using default converter
16164 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.handler.component.SuggestComponent – Initializing
SuggestComponent
16164 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.spelling.suggest.SolrSuggester – init:
{name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
16164 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.spelling.suggest.SolrSuggester – Dictionary loaded with
params:
{name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
16164 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.common.cloud.ZkStateReader – Load collection config
from:/collections/solr6606
16166 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.common.cloud.ZkStateReader – path=/collections/solr6606
configName=solr6606 specified config exists in ZooKeeper
16186 [coreLoadExecutor-5-thread-2] INFO
org.apache.solr.handler.ReplicationHandler – Commits will be reserved for
10000
16187 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener sending requests to
Searcher@6b1b4cac[solr6606_shard1_replica1]
main{StandardDirectoryReader(segments_5q:2567:nrt _9g(4.10.2):C794000
_io(4.10.2):C760000 _sf(4.10.2):C796000 _yk(4.10.2):C543000 _ze(4.10.2):C72000
_zf(4.10.2):C1000 _zg(4.10.2):C2000 _zh(4.10.2):C3000 _zi(4.10.2):C3000
_zj(4.10.2):C2121 _zk(4.10.2):C2879 _zl(4.10.2):C3000 _zm(4.10.2):C2000)}
16226 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore –
[solr6606_shard1_replica1] webapp=null path=null
params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false}
hits=0 status=0 QTime=39
16227 [searcherExecutor-7-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] webapp=null path=null
params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false}
hits=0 status=0 QTime=66
16227 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
16227 [searcherExecutor-7-thread-1] INFO org.apache.solr.core.SolrCore –
QuerySenderListener done.
16227 [searcherExecutor-6-thread-1] INFO
org.apache.solr.handler.component.SpellCheckComponent – Loading spell index
for spellchecker: default
16227 [searcherExecutor-7-thread-1] INFO
org.apache.solr.handler.component.SpellCheckComponent – Loading spell index
for spellchecker: default
16228 [searcherExecutor-6-thread-1] INFO
org.apache.solr.handler.component.SpellCheckComponent – Loading spell index
for spellchecker: wordbreak
16228 [searcherExecutor-7-thread-1] INFO
org.apache.solr.handler.component.SpellCheckComponent – Loading spell index
for spellchecker: wordbreak
16228 [searcherExecutor-6-thread-1] INFO
org.apache.solr.handler.component.SuggestComponent – Loading suggester index
for: mySuggester
16228 [searcherExecutor-7-thread-1] INFO
org.apache.solr.handler.component.SuggestComponent – Loading suggester index
for: mySuggester
16229 [searcherExecutor-7-thread-1] INFO
org.apache.solr.spelling.suggest.SolrSuggester – reload()
16228 [searcherExecutor-6-thread-1] INFO
org.apache.solr.spelling.suggest.SolrSuggester – reload()
16229 [searcherExecutor-6-thread-1] INFO
org.apache.solr.spelling.suggest.SolrSuggester – build()
16229 [searcherExecutor-7-thread-1] INFO
org.apache.solr.spelling.suggest.SolrSuggester – build()
16249 [searcherExecutor-7-thread-1] INFO org.apache.solr.core.SolrCore –
[collection1] Registered new searcher Searcher@2c8633e1[collection1]
main{StandardDirectoryReader(segments_1:1:nrt)}
{code}
The SuggestComponent init's correctly. It's only when the firstSearcher event
gets fired that the spellchecker and the suggester gets built. This is where
the problem lies.
We should just comment out the {{<str name="q">static firstSearcher warming in
solrconfig.xml</str>}} query from the {{firstSearcher}} query list, just as how
the queries are commented out from the {{newSearcher}} event listener.
We should also add a comment here saying that adding queries here does the
following internally and would lead to longer load times.
1. Builds the suggester, spellchecker and any other components which register
itself to new searcher events.
2. When an entry contains a sort/facet/function query and the field is not an
docValues field then it UnInverts the fields and puts it into the FieldCache
3. Adds entries to the other Solr caches.
The multiple reload and build commands is because I have 2 cores on my machine.
> figure out why suggester causes slow startup - even when not used
> -----------------------------------------------------------------
>
> Key: SOLR-6845
> URL: https://issues.apache.org/jira/browse/SOLR-6845
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
>
> SOLR-6679 was filed to track the investigation into the following problem...
> {panel}
> The stock solrconfig provides a bad experience with a large index... start up
> Solr and it will spin at 100% CPU for minutes, unresponsive, while it
> apparently builds a suggester index.
> ...
> This is what I did:
> 1) indexed 10M very small docs (only takes a few minutes).
> 2) shut down Solr
> 3) start up Solr and watch it be unresponsive for over 4 minutes!
> I didn't even use any of the fields specified in the suggester config and I
> never called the suggest request handler.
> {panel}
> ..but ultimately focused on removing/disabling the suggester from the sample
> configs.
> Opening this new issue to focus on actually trying to identify the root
> problem & fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
