On 7/29/2021 2:38 PM, Pratik Patel wrote:
I have following in my solrconfig.xml<autoCommit> <maxTime>60000</maxTime> <openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> <maxTime>30000</maxTime> </autoSoftCommit> I am triggering many solrClient.add( collName. <docs>, commitWithin ) calls from my code and commtWithin=6hours Based on the above configuration, I expect a soft commit to happen every 30 seconds but I don't see that happening. When I hit a solr query after 10 mins of add operation, I don't see any documents in the solr admin UI. Solr Admin UI mentions 0 documents even after 10-20 mins of add operation. Solr version is 8.9.0. updateLog is enabled. managed-schema has _version_ field Does commitWithin override the autosoftCommit setting ?
There's a little bit of a misconception here. To say that it does a commit every N seconds is not precisely correct.
The autoSoftCommit setting starts a counter as soon as something gets indexed. When that counter hits 30 seconds, a soft commit is started. I'm not entirely sure what makes it start another 30 second timer, but I think that it's probably the indexing of additional documents after the previous commit is started. If the commit takes a lot longer than the autoSoftCommit timer, like 3 minutes or so to complete, then you could end up with multiple commits happening at the same time, but there is a short circuit in Solr for that -- maxWarmingSearchers. Solr will not allow additional searchers to be opened once that limit is reached. Creating and warming the searcher is the expensive part of a commit, which is why autoCommit typically has openSearcher set to false.
Based on what you're seeing, it does sound like commitWithin overrides autoSoftCommit, but I really have no idea whether that's true. You could try setting it to something like 5 minutes to see what that does without waiting six hours.
Thanks, Shawn
