[
https://issues.apache.org/jira/browse/SOLR-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-13913:
----------------------------------
Component/s: CDCR
> CDCR should limit TLOG growth
> -----------------------------
>
> Key: SOLR-13913
> URL: https://issues.apache.org/jira/browse/SOLR-13913
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
> Reporter: Erick Erickson
> Priority: Major
>
> CDCR uses TLOGs for a queueing mechanism. If the connection between DCs goes
> down for any reason and is not caught, the tlogs will grow forever, which can
> lead to disk full situations and all that entails.
> Aside from that problem, it's not clear that reprocessing a zillion updates
> is faster than a full replication anyway.
> Since the full-index replication was added, we can avoid runaway tlogs by
> somehow noticing we haven't been connected to the remote DC for a long time,
> purge the tlogs (keeping just enough for peer sync of course) and do a full
> index replication next time we do connect.
> This is pretty vague, I don't have a good idea of whether tlog size is the
> right metric, or some sort of time since last successful transmission, or the
> queue size or some combination of these and others. The point is simply that
> after some threshold was crossed, reset to a zero state and avoid the
> pitfalls of continuing to accumulate updates.
> I'd suggest these be tunable parameters defined in solrconfig.xml since I can
> imagine that terabyte-scale indexes should fall back to full-index
> replication more rarely than megabyte-scale indexes.
> This idea came up in discussions and I wanted to preserve the it in case
> someone wants to pursue it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
