[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244176#comment-16244176 ] ASF subversion and git services commented on SOLR-11003: Commit dcbcabbd54c0a21c69a891d0fe5955d190d4bfc0 in lucene-solr's branch refs/heads/branch_7x from [~steve_rowe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dcbcabb ] SOLR-11003: Do not import ImmutableMap from shaded junit4-ant jar > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Fix For: 7.2 > > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244177#comment-16244177 ] ASF subversion and git services commented on SOLR-11003: Commit b28ed8f0ef84100dea2410b2ef394ad93aded27d in lucene-solr's branch refs/heads/master from [~steve_rowe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b28ed8f ] SOLR-11003: Do not import ImmutableMap from shaded junit4-ant jar > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Fix For: 7.2 > > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240499#comment-16240499 ] ASF subversion and git services commented on SOLR-11003: Commit fdff8deabdffe6a840fc1cbb5eb249aaed3fed63 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fdff8de ] SOLR-11003: Improve description of the feature in the CHANGES file > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Fix For: 7.2 > > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240498#comment-16240498 ] ASF subversion and git services commented on SOLR-11003: Commit a0163232edfaa3068c6240a352781e0637f53b1c in lucene-solr's branch refs/heads/branch_7x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a016323 ] SOLR-11003: Improve description of the feature in the CHANGES file > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240493#comment-16240493 ] ASF subversion and git services commented on SOLR-11003: Commit a30c92a79cd75d3c4fe21829e70292f8314afd4f in lucene-solr's branch refs/heads/branch_7x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a30c92a ] SOLR-11003: Support bi-directional syncing of cdcr clusters. We still only support actively into one cluster cluster, but have the ability to switch indexing clusters and cdcr will replicate correctly > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238579#comment-16238579 ] ASF subversion and git services commented on SOLR-11003: Commit db18de7e6b9783df3bec80ca02de415e306c4e87 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=db18de7 ] SOLR-11003: Support bi-directional syncing of cdcr clusters. We still only support actively into one cluster cluster, but have the ability to switch indexing clusters and cdcr will replicate correctly > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker >Priority: Major > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227520#comment-16227520 ] Varun Thacker commented on SOLR-11003: -- I still don't see the test using two shards in this patch > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227317#comment-16227317 ] Varun Thacker commented on SOLR-11003: -- Sorry for the noise here. This was a local stale commit which got pushed with another commit and it wasn't intended. I plan on applying the latest patch that Amrit provided, testing that and then commit it. > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227315#comment-16227315 ] ASF subversion and git services commented on SOLR-11003: Commit 12d9d67dab3ae0619b92953edde8d4fec96786ff in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=12d9d67 ] Revert "SOLR-11003" This reverts commit bc46de3b2a07b77d40e85e1609d0ee04eacbcf76. > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227304#comment-16227304 ] ASF subversion and git services commented on SOLR-11003: Commit bc46de3b2a07b77d40e85e1609d0ee04eacbcf76 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc46de3 ] SOLR-11003 > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225107#comment-16225107 ] Varun Thacker commented on SOLR-11003: -- bq. 2 shards instead of 1? The only problem / caveat is the test will become heavier with two separate clusters running already, I hope it don't introduce unnecessary zk-timeouts or such. We'll find out. But I'd be more comfortable testing with 2 shards instead of 1. bq. Also removing CdcrQueue status (it causes no harm though). Sure then let's keep it then. We can add a code comment clarifying that this is only to observe the queue length when the test runs > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225077#comment-16225077 ] Amrit Sarkar commented on SOLR-11003: - [~varunthacker] Yes it applies smoothly. Do you need any changes in the current patch? The above two you mentioned, * 2 shards instead of 1? The only problem / caveat is the test will become heavier with two separate clusters running already, I hope it don't introduce unnecessary zk-timeouts or such. * Also removing CdcrQueue status (it causes no harm though). Let me know. > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, sample-configs.zip > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16223729#comment-16223729
]
Amrit Sarkar commented on SOLR-11003:
-
> 1. What are we trying to test with this part?
Print the current status of the cdcr -queue, tlog size, total tlogs. No usage
when all goes well, when it doesn't; it will provide us exact number of what's
up in the cdcr queues of the respective clusters, which cluster was behind etc.
> 2. We should change the collections to be 2 shards instead of 1. It's
> probably have better test coverage for DBQs etc.
Makes sense. Though it is proven there is no multiple back and forth DBQ, as we
add the same doc in next iteration which we delete via DBQ.
Also, your latest patch, I cannot apply on the {{master}} branch. First, the
patch is cut from {{solr}} module. Second, I think something's wrong with the
refactoring I did moving all cdcr tests under one directory.
> Enabling bi-directional CDCR on cluster for better failover
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch,
> SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch,
> SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch,
> sample-configs.zip
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time (given the backlog
> of replicating updates to other data center). ClusterACollectionA =>
> ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA.
> The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1665#comment-1665
]
Amrit Sarkar commented on SOLR-11003:
-
Varun,
thank you for the constructive feedback.
> We could however argue that this improvement could be implemented for the
> regular transaction log as well. So let's put this for another Jira
I understand what you are saying, we can certainly do this to make the
back-compat safe. This can be part of next iteration of CDCR optimisation.
> In CDCRBiDirectionalTest can we have one try block instead of two currently.
> In the finally we could do a null check before shutting down
I was following {{CdcrBootstrapTest}} format Shalin wrote, but yeah maybe two
try catch not required, changed as suggested.
> Methods like cdcrStart are the same in CdcrBootstrapTest. Can we put them in
> a util class and reuse?We also have
> BaseCdcrDistributedZkTest#invokeCdcrAction . Let's refactor the usage
> waitForClusterToSync is the same as CdcrBootsrapTest#waitForTargetToSync
Positively, moved to {{CdcrTestsUtil}}, we can surely make some code effective
refactoring when we write this.
> Even CdcrBootstrapTest#indexDocs can be factored into the util class and
> reused here?
We can, but then we have to specify the {{prefix}} for id for each collection,
we can leave the refactoring there I guess.
> Can we use the logger instead of System.out.println
Done.
> After we add docs to cluster 1, can we do a assert that the numFound is not 0
> . Just a sanity check to make sure we indexed docs atleast.
We have already added this check at every stage, doing {{*:*}} at source.
> Are we printing the queue responses just for debugging? can we change that to
> an assert?
We cannot assert on that, as queue responses will be different for different
clusters, we are just printing it for our reference.
> For /get?getVersions=X we need to do distrib=false . But since this is a one
> shard collection we don't need to I guess. You can add fingerprint=true and
> use the maxVersionEncountered key instead of calculating the max in code?
I have to see this, how to do this part. I copied _blindly_ from
{{CdcrBootstrapTest}}, and as {{fingerprint}} was not introduced when bootstrap
came in, its not there.
> After a DBQ/delete-by-id/atomic updates we do a 2s thread wait. Can we not
> rely on this and use waitForClusterToSync? In general we should not be doing
> any thread sleeps in the test
Yeah that was poor test writing, fixed this. Thanks for pointing it out.
Pending: introducing {{fingerprint}} to get the maxVersion.
> Enabling bi-directional CDCR on cluster for better failover
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch,
> SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch,
> SOLR-11003.patch, SOLR-11003.patch, sample-configs.zip
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time (given the backlog
> of replicating updates to other data center). ClusterACollectionA =>
> ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA.
> The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221766#comment-16221766
]
Varun Thacker commented on SOLR-11003:
--
Hi Amrit,
Patch looks great!
{{isTargetCluster}} uses the entry list size to see if we have the extra
information in the tlog. This works and handles back-compat correctly.
However I'm more inclined to do this:
Encode a CDCR TLOG version if each entry . We can use this for the back-compat
check. This potentially opens the door for new features down the road to be
handled differently, when to stop caring about old back-compat checks ( next
major version ) etc . Lucene does it from 7.0 ( LUCENE-7703 )
We could however argue that this improvement could be implemented for the
regular transaction log as well. So let's put this for another Jira
- In CDCRBiDirectionalTest can we have one try block instead of two currently.
In the finally we could do a null check before shutting down
- Methods like {{cdcrStart}} are the same in CdcrBootstrapTest. Can we put them
in a util class and reuse?We also have
{{BaseCdcrDistributedZkTest#invokeCdcrAction}} . Let's refactor the usage
- {{waitForClusterToSync}} is the same as
{{CdcrBootsrapTest#waitForTargetToSync}}
- Even {{CdcrBootstrapTest#indexDocs}} can be factored into the util class and
reused here?
- Can we use the logger instead of System.out.println
- After we add docs to cluster 1, can we do a assert that the numFound is not 0
. Just a sanity check to make sure we indexed docs atleast.
- Are we printing the queue responses just for debugging? can we change that to
an assert?
- For {{/get?getVersions=X}} we need to do distrib=false . But since this is a
one shard collection we don't need to I guess. You can add fingerprint=true and
use the maxVersionEncountered key instead of calculating the max in code?
- After a DBQ/delete-by-id/atomic updates we do a 2s thread wait. Can we not
rely on this and use waitForClusterToSync? In general we should not be doing
any thread sleeps in the test
> Enabling bi-directional CDCR on cluster for better failover
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: SOLR-11003-tlogutils.patch, SOLR-11003.patch,
> SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch,
> SOLR-11003.patch, sample-configs.zip
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time (given the backlog
> of replicating updates to other data center). ClusterACollectionA =>
> ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA.
> The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184283#comment-16184283 ] Amrit Sarkar commented on SOLR-11003: - Documentation changes for CDCR jira: SOLR-11412 > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR on cluster for better failover
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183564#comment-16183564 ] Amrit Sarkar commented on SOLR-11003: - Working on documentation changes now. Will open a new JIRA as changes needs to be discussed in detail. > Enabling bi-directional CDCR on cluster for better failover > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time (given the backlog > of replicating updates to other data center). ClusterACollectionA => > ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. > The STRONG RECOMMENDED way to keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
