[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download tlog's from source

2017-10-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16219603#comment-16219603
 ] 

ASF subversion and git services commented on SOLR-11326:


Commit 914c42ab0da852c75b367f2826f45b41ef994cea in lucene-solr's branch 
refs/heads/branch_7x from [~varunthacker]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=914c42a ]

SOLR-11326: A bootstrap of a target cluster does not need to download the tlog 
files from the source cluster

(cherry picked from commit cb20bbd)


> CDCR bootstrap should not download tlog's from source
> -
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
> Fix For: 7.2
>
> Attachments: SOLR-11326.patch, SOLR-11326.patch, SOLR-11326.patch, 
> SOLR-11326.patch, SOLR-11326.patch, SOLR-11326.patch, WITHOUT-FIX.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap 
> the tlog's from the source is getting download
> snippet1:
> {code}
>[junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
>[junit4]   2> 42934 INFO  
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr 
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) 
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to 
> bootstrap target collection: cdcr-target shard: shard1 leader: 
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>[junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr 
> path=/replication 
> params={qt=/replication=javabin=2=indexversion} status=0 
> QTime=0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 
> 1503514968639
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication 
> process
>[junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, 
> name=tlog.000.1576549701811961856}, {size=4770, 
> name=tlog.001.1576549702515556352}, {size=4770, 
> name=tlog.002.1576549702628802560}, {size=4770, 
> name=tlog.003.1576549702720028672}, {size=4770, 
> name=tlog.004.1576549702799720448}, {size=4770, 
> name=tlog.005.1576549702894092288}, {size=4770, 
> name=tlog.006.1576549703029358592}, {size=4770, 
> name=tlog.007.1576549703126876160}, {size=4770, 
> name=tlog.008.1576549703208665088}, {size=4770, 
> name=tlog.009.1576549703295696896}
> {code}
> snippet2:
> {code}
>  17070[junit4]   2> 677606 INFO  (qtp22544544-5725) [] 
> o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: 
> cdcr-target, shard: shard1^M
>  17071[junit4]   2> 677608 INFO  

[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download tlog's from source

2017-10-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16219555#comment-16219555
 ] 

ASF subversion and git services commented on SOLR-11326:


Commit cb20bbdd1fb6bde7edf00d397dc6a8dc200d40eb in lucene-solr's branch 
refs/heads/master from [~varunthacker]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb20bbd ]

SOLR-11326: A bootstrap of a target cluster does not need to download the tlog 
files from the source cluster


> CDCR bootstrap should not download tlog's from source
> -
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch, SOLR-11326.patch, 
> SOLR-11326.patch, SOLR-11326.patch, SOLR-11326.patch, WITHOUT-FIX.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap 
> the tlog's from the source is getting download
> snippet1:
> {code}
>[junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
>[junit4]   2> 42934 INFO  
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr 
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) 
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to 
> bootstrap target collection: cdcr-target shard: shard1 leader: 
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>[junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr 
> path=/replication 
> params={qt=/replication=javabin=2=indexversion} status=0 
> QTime=0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 
> 1503514968639
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication 
> process
>[junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, 
> name=tlog.000.1576549701811961856}, {size=4770, 
> name=tlog.001.1576549702515556352}, {size=4770, 
> name=tlog.002.1576549702628802560}, {size=4770, 
> name=tlog.003.1576549702720028672}, {size=4770, 
> name=tlog.004.1576549702799720448}, {size=4770, 
> name=tlog.005.1576549702894092288}, {size=4770, 
> name=tlog.006.1576549703029358592}, {size=4770, 
> name=tlog.007.1576549703126876160}, {size=4770, 
> name=tlog.008.1576549703208665088}, {size=4770, 
> name=tlog.009.1576549703295696896}
> {code}
> snippet2:
> {code}
>  17070[junit4]   2> 677606 INFO  (qtp22544544-5725) [] 
> o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: 
> cdcr-target, shard: shard1^M
>  17071[junit4]   2> 677608 INFO  (qtp22544544-5725) [] 
> o.a.s.h.CdcrReplicatorManager Submitting 

[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download tlog's from source

2017-10-18 Thread Amrit Sarkar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209219#comment-16209219
 ] 

Amrit Sarkar commented on SOLR-11326:
-

[~varunthacker],

Roger that!

I have uploaded two patches:
a) WITHOUT FIX and a test-method in {{CdcrBootstrapTest}} to prove tlogs get 
copied over to {{target}} when Bootstrap is completed.
b) WITH FIX and test-method in {{CdcrBootstrapTest}} to prove tlogs DOESN'T get 
copied over to {{target}} when Bootstrap is completed.

Apart from the changes, I fixed the log lines in {{CdcrBootstrapTest}}, which 
was misleading.

PLEASE NOTE: I have wrote the test on top of {{master}} branch and we have 
{{debugging}} code from SOLR-11467. If we remove the debugging code and apply 
this patch, it won't work. If we commit this and then try to remove SOLR-11467 
debugging code, that patch won't work. Please let me know if we need to handle 
this in better way or I need to write the patch without SOLR-11467 debugging 
patch.

> CDCR bootstrap should not download tlog's from source
> -
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch, SOLR-11326.patch, 
> WITHOUT-FIX.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap 
> the tlog's from the source is getting download
> snippet1:
> {code}
>[junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
>[junit4]   2> 42934 INFO  
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr 
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) 
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to 
> bootstrap target collection: cdcr-target shard: shard1 leader: 
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>[junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr 
> path=/replication 
> params={qt=/replication=javabin=2=indexversion} status=0 
> QTime=0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 
> 1503514968639
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication 
> process
>[junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, 
> name=tlog.000.1576549701811961856}, {size=4770, 
> name=tlog.001.1576549702515556352}, {size=4770, 
> name=tlog.002.1576549702628802560}, {size=4770, 
> name=tlog.003.1576549702720028672}, {size=4770, 
> name=tlog.004.1576549702799720448}, {size=4770, 
> name=tlog.005.1576549702894092288}, {size=4770, 
> name=tlog.006.1576549703029358592}, {size=4770, 
> name=tlog.007.1576549703126876160}, 

[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download tlog's from source

2017-10-17 Thread Varun Thacker (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208257#comment-16208257
 ] 

Varun Thacker commented on SOLR-11326:
--

bq.  So why even have the option?

Looking at {{CdcrReplicationHandlerTest}} I see when this is used. When the 
source replica comes up , it needs all the transaction logs from the leader in 
case of leader failure. This also explains why this code exists with default as 
"true"

{code}
// fetch list of tlog files only if cdcr is activated
if (solrParams.getBool(TLOG_FILES, true) && 
core.getUpdateHandler().getUpdateLog() != null
&& core.getUpdateHandler().getUpdateLog() instanceof CdcrUpdateLog) {
{code}

So I think the patch is valid. 


[~sarkaramr...@gmail.com] can we add a test to make sure this functionality 
doesn't break in the future? Maybe we could use a similar mechanism as 
{{CdcrReplicationHandlerTest#assertUpdateLogsEquals}} to make sure that a 
bootstrap doesn't download any tlogs in the target. 

Let's add a test and then we can wrap up this Jira

> CDCR bootstrap should not download tlog's from source
> -
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap 
> the tlog's from the source is getting download
> snippet1:
> {code}
>[junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
>[junit4]   2> 42934 INFO  
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr 
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) 
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to 
> bootstrap target collection: cdcr-target shard: shard1 leader: 
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>[junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr 
> path=/replication 
> params={qt=/replication=javabin=2=indexversion} status=0 
> QTime=0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 
> 1503514968639
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication 
> process
>[junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, 
> name=tlog.000.1576549701811961856}, {size=4770, 
> name=tlog.001.1576549702515556352}, {size=4770, 
> name=tlog.002.1576549702628802560}, {size=4770, 
> name=tlog.003.1576549702720028672}, {size=4770, 
> name=tlog.004.1576549702799720448}, {size=4770, 
> name=tlog.005.1576549702894092288}, {size=4770, 
> name=tlog.006.1576549703029358592}, {size=4770, 
> name=tlog.007.1576549703126876160}, {size=4770, 
> 

[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download tlog's from source

2017-10-17 Thread Varun Thacker (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208245#comment-16208245
 ] 

Varun Thacker commented on SOLR-11326:
--

Is there any scenario where we need to download the tlogs? I don't see any 
use-case in master-slave / solrcloud or in cdcr . So why even have the option? 

Today in CDCR we explicitly ask the replication handler not to download the 
tlogs by doing this

{[solrParams.set(ReplicationHandler.TLOG_FILES, false);}}

But the param is not respected. The patch actually fixes the issue. But why 
even keep such an option when we don't need it?

Does someone know the motivation of this in {{ReplicationHander::getFileList}}

{code}
// fetch list of tlog files only if cdcr is activated
if (solrParams.getBool(TLOG_FILES, true) && 
core.getUpdateHandler().getUpdateLog() != null
&& core.getUpdateHandler().getUpdateLog() instanceof CdcrUpdateLog) {
{code}

> CDCR bootstrap should not download tlog's from source
> -
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap 
> the tlog's from the source is getting download
> snippet1:
> {code}
>[junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
>[junit4]   2> 42934 INFO  
> (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr 
> x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) 
> [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 
> x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to 
> bootstrap target collection: cdcr-target shard: shard1 leader: 
> http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>[junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.c.S.Request [cdcr-source_shard1_replica1]  webapp=/solr 
> path=/replication 
> params={qt=/replication=javabin=2=indexversion} status=0 
> QTime=0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 
> 1503514968639
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
>[junit4]   2> 43004 INFO  
> (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr 
> x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) 
> [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 
> x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication 
> process
>[junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr 
> c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] 
> o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, 
> name=tlog.000.1576549701811961856}, {size=4770, 
> name=tlog.001.1576549702515556352}, {size=4770, 
> name=tlog.002.1576549702628802560}, {size=4770, 
> name=tlog.003.1576549702720028672}, {size=4770, 
> name=tlog.004.1576549702799720448}, {size=4770, 
> name=tlog.005.1576549702894092288}, {size=4770, 
> name=tlog.006.1576549703029358592}, {size=4770, 
> name=tlog.007.1576549703126876160}, {size=4770, 
> name=tlog.008.1576549703208665088}, {size=4770, 
> name=tlog.009.1576549703295696896}
> {code}
>