[
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208257#comment-16208257
]
Varun Thacker commented on SOLR-11326:
--------------------------------------
bq. So why even have the option?
Looking at {{CdcrReplicationHandlerTest}} I see when this is used. When the
source replica comes up , it needs all the transaction logs from the leader in
case of leader failure. This also explains why this code exists with default as
"true"
{code}
// fetch list of tlog files only if cdcr is activated
if (solrParams.getBool(TLOG_FILES, true) &&
core.getUpdateHandler().getUpdateLog() != null
&& core.getUpdateHandler().getUpdateLog() instanceof CdcrUpdateLog) {
{code}
So I think the patch is valid.
[~sarkaramr...@gmail.com] can we add a test to make sure this functionality
doesn't break in the future? Maybe we could use a similar mechanism as
{{CdcrReplicationHandlerTest#assertUpdateLogsEquals}} to make sure that a
bootstrap doesn't download any tlogs in the target.
Let's add a test and then we can wrap up this Jira
> CDCR bootstrap should not download tlog's from source
> -----------------------------------------------------
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap
> the tlog's from the source is getting download
> snippet1:
> {code}
> [junit4] 2> 42931 INFO (qtp1525032019-69) [n:127.0.0.1:53178_solr
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1]
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
> [junit4] 2> 42934 INFO
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1)
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to
> bootstrap target collection: cdcr-target shard: shard1 leader:
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
> [junit4] 2> 43003 INFO (qtp1525032019-69) [n:127.0.0.1:53178_solr
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1]
> o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr
> path=/replication
> params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0
> QTime=0
> [junit4] 2> 43004 INFO
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
> [junit4] 2> 43004 INFO
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version:
> 1503514968639
> [junit4] 2> 43004 INFO
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
> [junit4] 2> 43004 INFO
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
> [junit4] 2> 43004 INFO
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication
> process
> [junit4] 2> 43041 INFO (qtp1525032019-71) [n:127.0.0.1:53178_solr
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1]
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649,
> name=tlog.0000000000000000000.1576549701811961856}, {size=4770,
> name=tlog.0000000000000000001.1576549702515556352}, {size=4770,
> name=tlog.0000000000000000002.1576549702628802560}, {size=4770,
> name=tlog.0000000000000000003.1576549702720028672}, {size=4770,
> name=tlog.0000000000000000004.1576549702799720448}, {size=4770,
> name=tlog.0000000000000000005.1576549702894092288}, {size=4770,
> name=tlog.0000000000000000006.1576549703029358592}, {size=4770,
> name=tlog.0000000000000000007.1576549703126876160}, {size=4770,
> name=tlog.0000000000000000008.1576549703208665088}, {size=4770,
> name=tlog.0000000000000000009.1576549703295696896}
> {code}
> snippet2:
> {code}
> 17070 [junit4] 2> 677606 INFO (qtp22544544-5725) [ ]
> o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection:
> cdcr-target, shard: shard1^M
> 17071 [junit4] 2> 677608 INFO (qtp22544544-5725) [ ]
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor^M
> 17091 [junit4] 2> 677627 INFO (qtp22544544-5724) [ ]
> o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr
> path=/replication
> params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0
> QTime=0^M
> 17092 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Master's generation: 12^M
> 17093 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Master's version: 1504127155366^M
> 17094 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Slave's generation: 1^M
> 17095 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Slave's version: 0^M
> 17096 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Starting replication process^M
> 17097 [junit4] 2> 677654 INFO (qtp22544544-5725) [ ]
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4773,
> name=tlog.0000000000000000009}]^M
> 17098 [junit4] 2> 677655 INFO (qtp22544544-5725) [ ]
> o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr
> path=/replication
> params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist}
> status=0 QTime=26^M
> 17099 [junit4] 2> 677656 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Number of files in latest index in master: 23^M
> 17100 [junit4] 2> 677656 INFO (recoveryExecutor-1024-thread-1) [ ]
> o.a.s.h.IndexFetcher Number of tlog files in master: 1^M
> {code}
> Also here's the code comment which also says we don't need to download the
> tlog's
> {code:title=CdcrRequestHandler.java|borderStyle=solid}
> ModifiableSolrParams solrParams = new ModifiableSolrParams();
> solrParams.set(ReplicationHandler.MASTER_URL, masterUrl);
> // we do not want the raw tlog files from the source
> solrParams.set(ReplicationHandler.TLOG_FILES, false);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
