[
https://issues.apache.org/jira/browse/SOLR-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117807#comment-14117807
]
Yonik Seeley commented on SOLR-6460:
------------------------------------
Some of the current tlog/peer-sync code won't work well with increased limits.
One simple example is peer-sync sends all the docs in a single message
(non-streaming)... obviously bad when the number of docs grows (guaranteed to
result in an OOM exception). This is why this issue is separate from raising
the peer-sync limit.
IIRC, another area of concern is that tlogs are kept open until they are
deleted (and thus we'll exhaust file descriptors).
Other code (like peersync code) needs to be audited to ensure that even if we
keep gigabytes of log files around, they won't all be read during a peersync
request (i.e. peersync limit needs to be decoupled from tlog lifetime). Some
of this issue is just ensuring that no bad things happen when we drastically
increase the tlog lifetime limit.
> Keep transaction logs around longer
> -----------------------------------
>
> Key: SOLR-6460
> URL: https://issues.apache.org/jira/browse/SOLR-6460
> Project: Solr
> Issue Type: Sub-task
> Reporter: Yonik Seeley
>
> Transaction logs are currently deleted relatively quickly... but we need to
> keep them around much longer to be used as a source for cross-datacenter
> recovery. This will also be useful in the future for enabling peer-sync to
> use more historical updates before falling back to replication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
