On 2/9/2018 9:29 AM, Webster Homer wrote:
A little more background. Our production Solrclouds are populated via CDCR,
CDCR does not replicate commits, Commits to the target clouds happen via
autoCommit settings
We see relvancy scores get inconsistent when there are too many deletes
which seems to happen when hard commits don't happen.
On Fri, Feb 9, 2018 at 10:25 AM, Webster Homer <webster.ho...@sial.com>
wrote:
I we do have autoSoftcommit set to 3 seconds. It is NOT the visibility of
the records that is my primary concern. I am concerned about is the
accumulation of uncommitted tlog files and the larger number of deleted
documents.
For the deleted documents: Have you ever done an optimize on the
collection? If so, you're going to need to re-do the optimize regularly
to keep deleted documents from growing out of control. See this issue
for a very technical discussion about it:
https://issues.apache.org/jira/browse/LUCENE-7976
Deleted documents probably aren't really related to what we've been
discussing. That shouldn't really be strongly affected by commit settings.
-----
A 3 second autoSoftCommit is VERY aggressive. If your soft commits are
taking longer than 3 seconds to complete, which is often what happens,
then that will lead to problems. I wouldn't expect it to cause the
kinds of problems you describe, though. It would manifest as Solr
working too hard, logging warnings or errors, and changes taking too
long to show up.
Assuming that the config for autoSoftCommit doesn't have the typo that
Erick mentioned.
----
I have never used CDCR, so I know very little about it. But I have seen
reports on this mailing list saying that transaction logs never get
deleted when CDCR is configured.
Below is a link to a mailing list discussion related to CDCR not
deleting transaction logs. Looks like for it to work right a buffer
needs to be disabled, and there may also be problems caused by not
having a complete zkHost string in the CDCR config:
http://lucene.472066.n3.nabble.com/CDCR-how-to-deal-with-the-transaction-log-files-td4345062.html
Erick also mentioned this.
Thanks,
Shawn
