[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[
https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902935#comment-16902935
]
Lucene/Solr QA commented on SOLR-11724:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 3m 49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 3m 49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 3m 49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 55m
46s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 64m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-11724 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977021/SOLR-11724.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / ed137db |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/524/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/524/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
> -
>
> Key: SOLR-11724
> URL: https://issues.apache.org/jira/browse/SOLR-11724
> Project: Solr
> Issue Type: Bug
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Fix For: 7.3.1, 7.4, 8.0
>
> Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch,
> SOLR-11724.patch, SOLR-11724.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Please find the discussion on:
> http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html
> If we index significant documents in to Source, stop indexing and then start
> CDCR; bootstrapping only copies the index to leader node of shards of the
> collection, and followers never receive the documents / index until and
> unless atleast one document is inserted again on source; which propels to
> target and target collection trigger index replication to followers.
> This behavior needs to be addressed in proper manner, either at target
> collection or while bootstrapping.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902876#comment-16902876 ] Shalin Shekhar Mangar commented on SOLR-11724: -- I just noticed the linked issue SOLR-13141 so we can use that and let this be as it is today. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch, > SOLR-11724.patch, SOLR-11724.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902868#comment-16902868 ] Shalin Shekhar Mangar commented on SOLR-11724: -- A less brittle way would be to update the term for the leader once bootstrap finishes. This way the replicas will automatically go to recovery. The latest patch implements this idea. Tests pass. What's the best way to fix this issue? This issue was supposed to be fixed in 7.3 but the committed code didn't actually fix it. Should we reopen this issue or create a new one? > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch, > SOLR-11724.patch, SOLR-11724.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858464#comment-16858464 ] Alexandre Rafalovitch commented on SOLR-11724: -- This issue is marked as part of 7.3.1, but was it actually? It is the only issue as marked unfinished against the completed 7.3.1 release. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch, > SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849705#comment-16849705 ] Tim commented on SOLR-11724: So what's the process for this patch to make it into an update? > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch, > SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[
https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844254#comment-16844254
]
Lucene/Solr QA commented on SOLR-11724:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 45m
57s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-11724 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12969150/SOLR-11724.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / ec6ac97 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/398/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/398/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
> -
>
> Key: SOLR-11724
> URL: https://issues.apache.org/jira/browse/SOLR-11724
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Fix For: 7.3.1, 7.4, 8.0
>
> Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch,
> SOLR-11724.patch
>
>
> Please find the discussion on:
> http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html
> If we index significant documents in to Source, stop indexing and then start
> CDCR; bootstrapping only copies the index to leader node of shards of the
> collection, and followers never receive the documents / index until and
> unless atleast one document is inserted again on source; which propels to
> target and target collection trigger index replication to followers.
> This behavior needs to be addressed in proper manner, either at target
> collection or while bootstrapping.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843890#comment-16843890 ] Amrit Sarkar commented on SOLR-11724: - Attached test which works smooth, with the proposed code. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch, > SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843554#comment-16843554 ] Amrit Sarkar commented on SOLR-11724: - Thanks [~TimSolr] and numerous others on the mailing list for reporting. The correct way of solving this issue is to identify the correct base-url of Solr for the respective core we need to trigger REQUESTRECOVERY to and create a local HttpSolrClient instead of using CloudSolrClient from CdcrReplicatorState (which will forward the request to the leader of the shard instead to the rightful solr node). I baked a small patch a few weeks back, still need to work on tests properly to see why it is failing now. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765096#comment-16765096 ] Tim commented on SOLR-11724: This fix seems to be bugged in 7.5. https://issues.apache.org/jira/browse/SOLR-13141 > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.3.1, 7.4, 8.0 > > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451682#comment-16451682 ] ASF subversion and git services commented on SOLR-11724: Commit 8fa7687413558b3bc65cbbbeb722a21314187e6a in lucene-solr's branch refs/heads/branch_7_3 from [~varun_saxena] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8fa7687 ] SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync with the leader > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Fix For: 7.4, master (8.0), 7.3.1 > > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439757#comment-16439757 ] Varun Thacker commented on SOLR-11724: -- Keeping it open. I want to backport this for 7.3.1 > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438032#comment-16438032 ] ASF subversion and git services commented on SOLR-11724: Commit b0c095c1804149bf558064d7f7df76f318a5c5ee in lucene-solr's branch refs/heads/branch_7x from [~varun_saxena] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b0c095c ] SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync with the leader (cherry picked from commit 93f9a65) > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438022#comment-16438022 ] ASF subversion and git services commented on SOLR-11724: Commit 93f9a65b1c8aa460489fdce50ed84d18168b53ef in lucene-solr's branch refs/heads/master from [~varun_saxena] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=93f9a65 ] SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync with the leader > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[
https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437010#comment-16437010
]
Amrit Sarkar commented on SOLR-11724:
-
Thanks Varun for the valuable review and feedback.
1. Yeah, I always resist using Map.functions, as they are not thread-safe,
should have done the same here, thanks for pin-pointing.
2. The test method was written for the review and specifically for SOLR-11724,
moved the test check to {{testConvertClusterToCdcrAndBootstrap}}.
3. I like using {{distrib=false}} instead of core API, huge +1 to the same.
I have incorporated all the necessary recommendations in the current patch,
thanks again.
> Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
> -
>
> Key: SOLR-11724
> URL: https://issues.apache.org/jira/browse/SOLR-11724
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch,
> SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch
>
>
> Please find the discussion on:
> http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html
> If we index significant documents in to Source, stop indexing and then start
> CDCR; bootstrapping only copies the index to leader node of shards of the
> collection, and followers never receive the documents / index until and
> unless atleast one document is inserted again on source; which propels to
> target and target collection trigger index replication to followers.
> This behavior needs to be addressed in proper manner, either at target
> collection or while bootstrapping.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[
https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436609#comment-16436609
]
Varun Thacker commented on SOLR-11724:
--
Hi Amrit,
Thanks for working on the patch! I like the solution here , seems like the
right way to solve the problem.
Here's some feedback on the patch:
# In CdcrReplicatorManager#sendRequestRecoveryToFollowers , the following line
looks dangerous (
[https://docs.oracle.com/javase/7/docs/api/java/util/Map.html#values()] )
##
{code:java}
replicas.remove(slice.getLeader());{code}
## Maybe we could add something like this in the inner for loop instead?
{code:java}
if (slice.getLeader().getCoreName().equals(replica.getCoreName())) {
continue;
}{code}
## Do we really need a separate test for this? Maybe in one of the existing
tests we could increase the target replicationFactor ?
## To assert doc counts , we have CdcrTestsUtil#waitForCoresToSync . How about
something like this instead?
{code:java}
protected static boolean assertShardInSync(String collection, String shard,
CloudSolrClient client) throws IOException, SolrServerException {
TimeOut waitTimeOut = new TimeOut(30, TimeUnit.SECONDS, TimeSource.NANO_TIME);
DocCollection docCollection =
client.getZkStateReader().getClusterState().getCollection(collection);
Slice correctSlice = null;
for (Slice slice : docCollection.getSlices()) {
if (shard.equals(slice.getName())) {
correctSlice = slice;
break;
}
}
assertNotNull(correctSlice);
long leaderDocCount;
try (HttpSolrClient leaderClient = new
HttpSolrClient.Builder(correctSlice.getLeader().getCoreUrl()).withHttpClient(client.getHttpClient()).build())
{
leaderDocCount = leaderClient.query(new
SolrQuery("*:*").setParam("distrib", "false")).getResults().getNumFound();
}
while (!waitTimeOut.hasTimedOut()) {
int replicasInSync = 0;
for (Replica replica : correctSlice.getReplicas()) {
try (HttpSolrClient leaderClient = new
HttpSolrClient.Builder(replica.getCoreUrl()).withHttpClient(client.getHttpClient()).build())
{
long replicaDocCount = leaderClient.query(new
SolrQuery("*:*").setParam("distrib", "false")).getResults().getNumFound();
if (replicaDocCount == leaderDocCount) replicasInSync++;
}
}
if (replicasInSync == correctSlice.getReplicas().size()) {
return true;
}
}
return false;
}{code}
> Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
> -
>
> Key: SOLR-11724
> URL: https://issues.apache.org/jira/browse/SOLR-11724
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch,
> SOLR-11724.patch, SOLR-11724.patch
>
>
> Please find the discussion on:
> http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html
> If we index significant documents in to Source, stop indexing and then start
> CDCR; bootstrapping only copies the index to leader node of shards of the
> collection, and followers never receive the documents / index until and
> unless atleast one document is inserted again on source; which propels to
> target and target collection trigger index replication to followers.
> This behavior needs to be addressed in proper manner, either at target
> collection or while bootstrapping.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[
https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384089#comment-16384089
]
Webster Homer commented on SOLR-11724:
--
This may be a different issue, but I see similar behavior with some replicas in
7.2.0
We have a number of replicas which have identical schemas. We found that TLOG
replicas give much more consistent search results.
We created a collection using TLOG replicas in our QA clouds.
We have a locally hosted solrcloud with 2 nodes, all our collections have 2
shards. We use CDCR to replicate the collections from this environment to 2
data centers hosted in Google cloud. This seems to work fairly well for our
collections with NRT replicas. However the new TLOG collection has problems.
The google cloud solrclusters have 4 nodes each (3 separate Zookeepers). 2
shards per collection with 2 replicas per shard.
We never see data show up in the tlog replica cloud collections, but we do see
tlog files show up on the cloud servers. I can see that all of the servers have
cdcr started, buffers are disabled.
The cdcr source configuration is:
"requestHandler":{"/cdcr":{
"name":"/cdcr",
"class":"solr.CdcrRequestHandler",
"replica":[
{
"zkHost":"[xxx-mzk01.sial.com:2181|http://xxx-mzk01.sial.com:2181/],[xxx-mzk02.sial.com:2181|http://xxx-mzk02.sial.com:2181/],[xxx-mzk03.sial.com:2181/solr|http://xxx-mzk03.sial.com:2181/solr];,
"source":"b2b-catalog-material-180124T",
"target":"b2b-catalog-material-180124T"},
{
"zkHost":"[-mzk01.sial.com:2181|http://-mzk01.sial.com:2181/],[-mzk02.sial.com:2181|http://-mzk02.sial.com:2181/],[-mzk03.sial.com:2181/solr|http://-mzk03.sial.com:2181/solr];,
"source":"b2b-catalog-material-180124T",
"target":"b2b-catalog-material-180124T"}],
"replicator":{
"threadPoolSize":4,
"schedule":500,
"batchSize":250},
"updateLogSynchronizer":\{"schedule":6
The target configurations in the 2 clouds are the same:
"requestHandler":{"/cdcr":{ "name":"/cdcr", "class":"solr.CdcrRequestHandler",
"buffer":{"defaultState":"disabled"}}}
-rw-r--r-- 1 apache apache 596517718 Feb 28 20:19
tlog.029.1593653529934823424
-rw-r--r-- 1 apache apache 647 Feb 28 22:18
tlog.030.1593670744554864641
[apache@uc1b-ecomqa-msc01 tlog]$ pwd
/var/solr/data/b2b-catalog-material-180124T_shard1_replica_t2/data/tlog
All of our collections have a timestamp field, index_date. In the source
collection all the records have a date of 2/28/2018 but the target collections
have a latest date of 1/26/2018
I don't see cdcr errors in the logs.
We have a number of similar collections that behave correctly. This is the only
collection that is a TLOG collection. It appears that CDCR doesn't support TLOG
collections.
We also see the same behavior in our production solrclouds. The collections
that use NRT replicas replicate fine with CDCR, the collection that uses TLOG
replicas do not. Several of the NRT collections have the same configurations as
the tlog colletion, so that seems to be the only difference between them
> Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
> -
>
> Key: SOLR-11724
> URL: https://issues.apache.org/jira/browse/SOLR-11724
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.1
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch,
> SOLR-11724.patch
>
>
> Please find the discussion on:
> http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html
> If we index significant documents in to Source, stop indexing and then start
> CDCR; bootstrapping only copies the index to leader node of shards of the
> collection, and followers never receive the documents / index until and
> unless atleast one document is inserted again on source; which propels to
> target and target collection trigger index replication to followers.
> This behavior needs to be addressed in proper manner, either at target
> collection or while bootstrapping.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370084#comment-16370084 ] Amrit Sarkar commented on SOLR-11724: - First patch uploaded with tests and effective solution, using RequestRecovery. Feedback will be appreciated. WIP. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349816#comment-16349816 ] Amrit Sarkar commented on SOLR-11724: - [~WebHomer], Can you elaborate you cluster details, number of shards, number of replicas. I am not able to comprehend the above statement of half data being searchable. If something use to work for 6.1 and not working for 7.2 is a concern. If the replicators are started successfully (CDCR=START), there is no reason whatsoever the data won't get replicated to target to either shards and won't be searchable after commit. Here, if not a single document is indexed in source after bootstrapping done on target. The replicas of the target collections which are non-leader will have empty index. To ignore this issue, having 1 replica each on target is the immediate fix, as one will not see empty indexes on either cores, let me know if the above doesn't make sense. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348769#comment-16348769 ] Webster Homer commented on SOLR-11724: -- We are seeing this behavior in 7.2.0. Sometimes the data shows up in a search, but when I look there are no index segments. It appears to be searching just tlogs. This looks like a critical issue to me. We observed the same behavior when we deleted all the data from a collection. On;y tlogs show up and only half are searchable. The configuration worked with Solr 6.1 but doesn't with Solr 7.2. We don't see any errors. We see logs of calls to /cdcr LASTPROCESSEDVERSION in the logs We don't just see this as a bootstrap issue either, we only see about half the data get replicated. I don't know if tlogs are present on the targets, as I don't have access to those file systems > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Priority: Major > Attachments: SOLR-11724.patch, SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348409#comment-16348409 ] Amrit Sarkar commented on SOLR-11724: - Attached patch replicating the issue on Bootstrap stated above. To trigger the index replication in followers, can we initial LIR manually? Requesting thoughts and feedback on solution design. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar >Priority: Major > Attachments: SOLR-11724.patch > > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast one document is inserted again on source; which propels to > target and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312939#comment-16312939 ] Amrit Sarkar commented on SOLR-11724: - [~shalinmangar]: checking in again to understand if the above is intended behavior and we can trigger Index Replication by follower nodes in Target collection once BS is done. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast document is inserted again on source; which propels to target > and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11724) Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target
[ https://issues.apache.org/jira/browse/SOLR-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278527#comment-16278527 ] Amrit Sarkar commented on SOLR-11724: - [~shalinmangar] wanted to check with you whether this is the intended behavior. > Cdcr Bootstrapping does not cause "index copying" to follower nodes on Target > - > > Key: SOLR-11724 > URL: https://issues.apache.org/jira/browse/SOLR-11724 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 7.1 >Reporter: Amrit Sarkar > > Please find the discussion on: > http://lucene.472066.n3.nabble.com/Issue-with-CDCR-bootstrapping-in-Solr-7-1-td4365258.html > If we index significant documents in to Source, stop indexing and then start > CDCR; bootstrapping only copies the index to leader node of shards of the > collection, and followers never receive the documents / index until and > unless atleast document is inserted again on source; which propels to target > and target collection trigger index replication to followers. > This behavior needs to be addressed in proper manner, either at target > collection or while bootstrapping. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
