bq: Most soft commit documentation talks about setting up soft commits with <maxtime> of about a second.
I think this is really a consequence of this being included in the example configs for illustrative purposes, personally I never liked this. There is no one right answer. I've seen soft commit intervals from -1 (never soft commit) to 1 second. The latter means most all of your caches are totally useless and might as well be turned off usually. What you haven't mentioned is how often you add new docs. Is it once a day? Steadily from 8:00 to 17:00? All in three hours in the morning? Whatever, your soft commit really should be longer than your autowarm interval. Configure autowarming to reference queries (firstSearcher or newSearcher events or autowarm counts in queryResultCache and filterCache. Say 16 in each of these latter for a start) such that they cause the external file to load. That _should_ prevent any queries from being blocked since the autowarming will happen in the background and while it's happening incoming queries will be served by the old searcher. Best, Erick On Fri, Oct 7, 2016 at 5:19 PM, Mike Lissner <mliss...@michaeljaylissner.com> wrote: > I have an index of about 4M documents with an external file field > configured to do boosting based on pagerank scores of each document. The > pagerank file is about 93MB as of today -- it's pretty big. > > Each day, I add about 1,000 new documents to the index, and I need them to > be available as soon as possible so that I can send out alerts to our users > about new content (this is Google Alerts, essentially). > > Soft commits seem to be exactly the thing for this, but whenever I open a > new searcher (which soft commits seem to do), the external file is > reloaded, and all queries are halted until it finishes loading. When I just > measured, this took about 30 seconds to complete. Most soft commit > documentation talks about setting up soft commits with <maxtime> of about a > second. > > Is there anything I can do to make the external file field not get reloaded > constantly? It only changes about once a month, and I want to use soft > commits to power the alerts feature. > > Thanks, > > Mike
