Thanks for the reply Shawn. I did an experiment where I set commitWithin to be 30 seconds. With commitWithin=30 seconds, I could see documents in Solr Admin UI and its count increasing at an expected interval. I still could not see the log messages in solr container for soft commits but may be that's related to log level settings in the new solr image of version 8.9.0 (I did an upgrade recently)
Based on this, it does seem like commitWithin overrides the autoSoftCommit interval ! To find it through experimentation is extremely painful though, sadly documentation does not mention this crucial detail. On Thu, Jul 29, 2021 at 5:13 PM Shawn Heisey <[email protected]> wrote: > On 7/29/2021 2:38 PM, Pratik Patel wrote: > > I have following in my solrconfig.xml > > > > <autoCommit> > > <maxTime>60000</maxTime> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > <autoSoftCommit> > > <maxTime>30000</maxTime> > > </autoSoftCommit> > > > > I am triggering many solrClient.add( collName. <docs>, commitWithin ) > calls > > from my code and commtWithin=6hours > > > > Based on the above configuration, I expect a soft commit to happen every > 30 > > seconds but I don't see that happening. When I hit a solr query after 10 > > mins of add operation, I don't see any documents in the solr admin UI. > Solr > > Admin UI mentions 0 documents even after 10-20 mins of add operation. > > > > Solr version is 8.9.0. > > updateLog is enabled. > > managed-schema has _version_ field > > > > Does commitWithin override the autosoftCommit setting ? > > There's a little bit of a misconception here. To say that it does a > commit every N seconds is not precisely correct. > > The autoSoftCommit setting starts a counter as soon as something gets > indexed. When that counter hits 30 seconds, a soft commit is started. > I'm not entirely sure what makes it start another 30 second timer, but I > think that it's probably the indexing of additional documents after the > previous commit is started. If the commit takes a lot longer than the > autoSoftCommit timer, like 3 minutes or so to complete, then you could > end up with multiple commits happening at the same time, but there is a > short circuit in Solr for that -- maxWarmingSearchers. Solr will not > allow additional searchers to be opened once that limit is reached. > Creating and warming the searcher is the expensive part of a commit, > which is why autoCommit typically has openSearcher set to false. > > Based on what you're seeing, it does sound like commitWithin overrides > autoSoftCommit, but I really have no idea whether that's true. You > could try setting it to something like 5 minutes to see what that does > without waiting six hours. > > Thanks, > Shawn > >
