[ 
https://issues.apache.org/jira/browse/SOLR-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496817#comment-15496817
 ] 

Russell Black commented on SOLR-9521:
-------------------------------------

Our production system running a 3.6 solr cluster does something similar.  The 
first searcher warms itself by using the cache from another running server in 
the cluster.  It works by providing a cache implementation that keeps track of 
the query params for each cache entry:

{code:xml}
    <filterCache class="com.footnote.solr.RemoteAutoWarmCache"
                 impl="solr.FastLRUCache"
                 size="512"
                 initialSize="512"
                 autowarmCount="150"/>

    <queryResultCache class="com.footnote.solr.RemoteAutoWarmCache"
                     impl="solr.LRUCache"
                     size="512"
                     initialSize="512"
                     autowarmCount="150"/>
{code}

Then there is a request handler that serves up the top search queries from the 
cache:

{code:xml}
   <requestHandler name="/remoteautowarm" 
class="com.footnote.solr.RemoteAutoWarmRequestHandler" />
{code}

And a firstSearcher listener that fetches the cache queries from the cluster's 
{{/remoteautowarm}} handler. 

{code:xml}
    <listener event="firstSearcher" 
class="com.footnote.solr.RemoteAutoWarmEventListener">
      <str name="url">http://10.100.25.25/solr/remoteautowarm</str>
      <arr name="fallback">
        <lst>
          <str name="q">static firstSearcher warming in solrconfig.xml</str>
        </lst>
      </arr>
    </listener>
{code}

I have been using this code in production on a 3.6 cluster for several years.  
If you are interested, I can share the code.

> Warm first searcher with queries from log file
> ----------------------------------------------
>
>                 Key: SOLR-9521
>                 URL: https://issues.apache.org/jira/browse/SOLR-9521
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: search
>    Affects Versions: 6.2
>            Reporter: Russell Black
>            Priority: Minor
>              Labels: performance
>         Attachments: LogFileWarmer.java
>
>
> The only way I know of to warm the first searcher before it is opened, is to 
> hard code warming queries in
> {code:xml}<listener event="firstSearcher" 
> class="solr.QuerySenderListener">{code}
> For our application, hard-coding is not a good solution since some of our 
> more expensive filter queries change periodically.  
> I have created a plugin that warms the first searcher from the most recent 
> entries in the log files.  In my opinion this capability ought to be part of 
> solr, so I'm contributing my code in the hopes that it can be incorporated.
> It works by wrapping an instance of QuerySenderListener, setting its 
> "queries" argument from recent queries from the log files.  It is configured 
> like this:
> {code:xml}
> <listener event="firstSearcher" class="solrplugin.LogFileWarmer">
>   <!-- The arguments shown below are the defaults and could be omitted -->
>   <!-- A list of filenames to search.  They are searched in the order
>        listed, and the lines in each file are searched in reverse
>        order, so that the most recent queries are used to warm the
>        cache.  Searching stops when it has gathered enough queries.
>   -->
>   <str name="file">logs/solr.log</str>
>   <str name="file">logs/solr.log.1</str>
>   <str name="file">logs/solr.log.2</str>
>   <str name="file">logs/solr.log.3</str>
>   <str name="file">logs/solr.log.4</str>
>   <str name="file">logs/solr.log.5</str>
>   <str name="file">logs/solr.log.6</str>
>   <str name="file">logs/solr.log.7</str>
>   <str name="file">logs/solr.log.8</str>
>   <str name="file">logs/solr.log.9</str>
>   
>   <!-- The maximum number of queries to use from logs -->
>   <int name="count">100</int>
>   <!-- The maximum number of log lines to search in order to find <count> 
> queries-->
>   <int name="maxLines">1000000</int>
>   <!-- The regular expression that matches the query string from the log 
> line.  It expects the query string to be in capture group 1. -->
>   <str name="regex">path=/select params=\{([^ ]+)\} </str>
>   <!-- if no queries can be found in the log files, fall back to this -->
>   <arr name="fallback">
>     <lst><str name="q">solr</str><str name="sort">price asc</str></lst>
>     <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>
>   </arr>
> <listener>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to