[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[
https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085858#comment-17085858
]
ASF subversion and git services commented on SOLR-14371:
Commit 59cffaa9c6a7702a6f3313266c88698dcbb899d2 in lucene-solr's branch
refs/heads/branch_8x from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=59cffaa ]
SOLR-14371 Zk StatusHandler should know about dynamic zk config (#1392)
(cherry picked from commit 03363f413f2134594b012175deb3f10ec9384400)
with java11 -> java8 modifications
> Zk StatusHandler should know about dynamic zk config
>
>
> Key: SOLR-14371
> URL: https://issues.apache.org/jira/browse/SOLR-14371
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, SolrCloud
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.6
>
> Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> With v3.5, Zookeeper supports [dynamic
> reconfig|https://zookeeper.apache.org/doc/r3.6.0/zookeeperReconfig.html].
> Solr will happily connect to an ensemble with dynamic reconfiguration
> enabled, but it is not "aware of it". The ZK Status screen does not display
> whether the ensemble is in reconfig mode and is not able to load the dynamic
> set of hosts from zk.
> This JIRA aims to make the ZookeeperStatusHandler able to display zk status
> as well as show status for all zk hosts in the ensemble, even ones that are
> not part of the static connection string that Solr is aware of. It will also
> show warnings if Solr's static zk connection string is out of sync with the
> truth.
> Currently Solr has a static ZK connection string and is not able to
> re-configure itself when zk cluster is reconfigured. Fixing this is not part
> of this issue but is deferred to SOLR-14389.
> I discovered this while experimenting with the [solr-operator for
> Kubernetes|https://github.com/bloomberg/solr-operator]. It configures a
> reconfigurable ZK ensemble and configures a single LB host:port as its zk
> connection string. This issue will flag a warning in this case since the
> connection string is different from the set of zk hosts in zk node
> {{/zookeeper/config}}. This warning can be removed once Solr fully supports
> dynamic reconfiguration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[
https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085813#comment-17085813
]
ASF subversion and git services commented on SOLR-14371:
Commit 03363f413f2134594b012175deb3f10ec9384400 in lucene-solr's branch
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=03363f4 ]
SOLR-14371 Zk StatusHandler should know about dynamic zk config (#1392)
> Zk StatusHandler should know about dynamic zk config
>
>
> Key: SOLR-14371
> URL: https://issues.apache.org/jira/browse/SOLR-14371
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> With v3.5, Zookeeper supports [dynamic
> reconfig|https://zookeeper.apache.org/doc/r3.6.0/zookeeperReconfig.html].
> Solr will happily connect to an ensemble with dynamic reconfiguration
> enabled, but it is not "aware of it". The ZK Status screen does not display
> whether the ensemble is in reconfig mode and is not able to load the dynamic
> set of hosts from zk.
> This JIRA aims to make the ZookeeperStatusHandler able to display zk status
> as well as show status for all zk hosts in the ensemble, even ones that are
> not part of the static connection string that Solr is aware of. It will also
> show warnings if Solr's static zk connection string is out of sync with the
> truth.
> Currently Solr has a static ZK connection string and is not able to
> re-configure itself when zk cluster is reconfigured. Fixing this is not part
> of this issue but is deferred to SOLR-14389.
> I discovered this while experimenting with the [solr-operator for
> Kubernetes|https://github.com/bloomberg/solr-operator]. It configures a
> reconfigurable ZK ensemble and configures a single LB host:port as its zk
> connection string. This issue will flag a warning in this case since the
> connection string is different from the set of zk hosts in zk node
> {{/zookeeper/config}}. This warning can be removed once Solr fully supports
> dynamic reconfiguration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[
https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076402#comment-17076402
]
Houston Putman commented on SOLR-14371:
---
+1
I like the new patch, and I think it's a great intermediary step until we can
get Solr to work with reconfiguration.
I did a bit of research into the ZK client libraries late last week to see why
the reconfiguration wasn't supported there. I didn't really find anything
except for the fact that the only HostProvider available is still the
{{StaticHostProvider}}, which merely loops through the hosts provided in the
connection string.
Thanks so much for doing all of this testing and research. I'll dive into the
solr-operator issue when I get the chance, but fixing that should be relatively
easier.
> Zk StatusHandler should know about dynamic zk config
>
>
> Key: SOLR-14371
> URL: https://issues.apache.org/jira/browse/SOLR-14371
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> With v3.5, Zookeeper supports [dynamic
> reconfig|https://zookeeper.apache.org/doc/r3.6.0/zookeeperReconfig.html].
> Solr will happily connect to an ensemble with dynamic reconfiguration
> enabled, but it is not "aware of it". The ZK Status screen does not display
> whether the ensemble is in reconfig mode and is not able to load the dynamic
> set of hosts from zk.
> This JIRA aims to make the ZookeeperStatusHandler able to display zk status
> as well as show status for all zk hosts in the ensemble, even ones that are
> not part of the static connection string that Solr is aware of. It will also
> show warnings if Solr's static zk connection string is out of sync with the
> truth.
> Currently Solr has a static ZK connection string and is not able to
> re-configure itself when zk cluster is reconfigured. Fixing this is not part
> of this issue but is deferred to SOLR-14389.
> I discovered this while experimenting with the [solr-operator for
> Kubernetes|https://github.com/bloomberg/solr-operator]. It configures a
> reconfigurable ZK ensemble and configures a single LB host:port as its zk
> connection string. This issue will flag a warning in this case since the
> connection string is different from the set of zk hosts in zk node
> {{/zookeeper/config}}. This warning can be removed once Solr fully supports
> dynamic reconfiguration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[
https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076245#comment-17076245
]
Jan Høydahl commented on SOLR-14371:
[~houston] what do you think about the current patch with the new warning? That
warning will then show up when using solr-operator, since Solr cannot know
whether the host:port in connection string is in fact a load-balancer or just
some alias for one of the zk hsots.
I also changed the issue description to match my current understanding of what
is going on.
> Zk StatusHandler should know about dynamic zk config
>
>
> Key: SOLR-14371
> URL: https://issues.apache.org/jira/browse/SOLR-14371
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> With v3.5, Zookeeper supports [dynamic
> reconfig|https://zookeeper.apache.org/doc/r3.6.0/zookeeperReconfig.html].
> Solr will happily connect to an ensemble with dynamic reconfiguration
> enabled, but it is not "aware of it". The ZK Status screen does not display
> whether the ensemble is in reconfig mode and is not able to load the dynamic
> set of hosts from zk.
> This JIRA aims to make the ZookeeperStatusHandler able to display zk status
> as well as show status for all zk hosts in the ensemble, even ones that are
> not part of the static connection string that Solr is aware of. It will also
> show warnings if Solr's static zk connection string is out of sync with the
> truth.
> Currently Solr has a static ZK connection string and is not able to
> re-configure itself when zk cluster is reconfigured. Fixing this is not part
> of this issue but is deferred to SOLR-14389.
> I discovered this while experimenting with the [solr-operator for
> Kubernetes|https://github.com/bloomberg/solr-operator]. It configures a
> reconfigurable ZK ensemble and configures a single LB host:port as its zk
> connection string. This issue will flag a warning in this case since the
> connection string is different from the set of zk hosts in zk node
> {{/zookeeper/config}}. This warning can be removed once Solr fully supports
> dynamic reconfiguration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[ https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075949#comment-17075949 ] Jan Høydahl commented on SOLR-14371: So I feel the current PR solves the intention of the issue, and at the same time warning users that Solr does not support dynamic reconfig yet, and I plan to commit this on Tuesday. I also spun off SOLR-14389 to investigate whether we can support dynamic reconfig for real. > Zk StatusHandler should know about dynamic zk config > > > Key: SOLR-14371 > URL: https://issues.apache.org/jira/browse/SOLR-14371 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png > > Time Spent: 50m > Remaining Estimate: 0h > > With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator > for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB > (Service) in front of all zookeepers, and the zkclient will then fetch list > of all zookeepers from special zknode /zookeeper/config and reconfigure > itself with connection to all zk nodes listed. So you can then scale up/down > number of zk nodes dynamically without restarting solr. > However, the Admin UI displays errors since it believes it is connected to > only one zk, which is contradictory to what zk itself reports. We need to > make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient > what current zkHost is instead of relying on Zk_HOST static setting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[ https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073569#comment-17073569 ] Jan Høydahl commented on SOLR-14371: I suggest as a first step, we handle this by letting the Admin UI display a clear warning if it detects that ZK connection string is different than the dynamic list of hosts. That way people are aware of the risks, until we fix Solr's zk code to actually be aware of dynamic reconfiguration. !dynamic-reconfig-warning.png|width=900! As that effort overlaps/collides with SOLR-14306, let's defer that part to that issue. I'll commit the proposed warning logic to the PR. > Zk StatusHandler should know about dynamic zk config > > > Key: SOLR-14371 > URL: https://issues.apache.org/jira/browse/SOLR-14371 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Attachments: dynamic-reconfig-warning.png, dynamic-reconfig.png > > Time Spent: 50m > Remaining Estimate: 0h > > With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator > for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB > (Service) in front of all zookeepers, and the zkclient will then fetch list > of all zookeepers from special zknode /zookeeper/config and reconfigure > itself with connection to all zk nodes listed. So you can then scale up/down > number of zk nodes dynamically without restarting solr. > However, the Admin UI displays errors since it believes it is connected to > only one zk, which is contradictory to what zk itself reports. We need to > make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient > what current zkHost is instead of relying on Zk_HOST static setting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[
https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073223#comment-17073223
]
Jan Høydahl commented on SOLR-14371:
The PR is now ready for broader review, for those interested.
Since embedded ZK and ZK in tests are hardcoded to a standalone variant, and
big code changes are needed to spin up a Quorum, there are no unit tests for
the new {{getConfig()}} method in {{SolrZkClient}}. I have tested manually with
an external zk cluster that it works though.
Here is a condensed summary of the current PR
* If zk quorum has dynamic reconfiguration, the ZkStatus handler and Admin UI
view will now display info for each of the currently active Zookeepers, even if
they differ from initial ZK_HOST
* That meanns you can configure ZK_HOST with a single LB address in front of
all ZKs (like Solr Operator for k8s) and it will "just work"
* New info in JSON response and UI: {{dynamicReconfig=true/false}}
* New info for each ZK host: {{role=participant/follower}}
> Zk StatusHandler should know about dynamic zk config
>
>
> Key: SOLR-14371
> URL: https://issues.apache.org/jira/browse/SOLR-14371
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator
> for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB
> (Service) in front of all zookeepers, and the zkclient will then fetch list
> of all zookeepers from special zknode /zookeeper/config and reconfigure
> itself with connection to all zk nodes listed. So you can then scale up/down
> number of zk nodes dynamically without restarting solr.
> However, the Admin UI displays errors since it believes it is connected to
> only one zk, which is contradictory to what zk itself reports. We need to
> make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient
> what current zkHost is instead of relying on Zk_HOST static setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[ https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071840#comment-17071840 ] Jan Høydahl commented on SOLR-14371: Ok, first PR draft is out https://github.com/apache/lucene-solr/pull/1392 Tries to unify the old parsing of ZK_HOST and parsing of dynamic config. Needs more testing. I'll try to test with the Solr Operator and see if we get info for all ZKs. Also needs some more work in UI for how to interpret leader vs follower vs participant vs observer as zk-node roles. Now we require exactly one leader and assume the rest are followers, but I think there are other allowed mixes too. > Zk StatusHandler should know about dynamic zk config > > > Key: SOLR-14371 > URL: https://issues.apache.org/jira/browse/SOLR-14371 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator > for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB > (Service) in front of all zookeepers, and the zkclient will then fetch list > of all zookeepers from special zknode /zookeeper/config and reconfigure > itself with connection to all zk nodes listed. So you can then scale up/down > number of zk nodes dynamically without restarting solr. > However, the Admin UI displays errors since it believes it is connected to > only one zk, which is contradictory to what zk itself reports. We need to > make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient > what current zkHost is instead of relying on Zk_HOST static setting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14371) Zk StatusHandler should know about dynamic zk config
[ https://issues.apache.org/jira/browse/SOLR-14371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069380#comment-17069380 ] Jan Høydahl commented on SOLR-14371: [~houston] FYI > Zk StatusHandler should know about dynamic zk config > > > Key: SOLR-14371 > URL: https://issues.apache.org/jira/browse/SOLR-14371 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Priority: Major > > With zk 3.5 it supports dynamic reconfig, which is used by the solr-operator > for Kubernetes. Then Solr is given a zkHost of one url pointing to a LB > (Service) in front of all zookeepers, and the zkclient will then fetch list > of all zookeepers from special zknode /zookeeper/config and reconfigure > itself with connection to all zk nodes listed. So you can then scale up/down > number of zk nodes dynamically without restarting solr. > However, the Admin UI displays errors since it believes it is connected to > only one zk, which is contradictory to what zk itself reports. We need to > make ZookeeperStatusHandler aware of dynamic reconfig so it asks zkclient > what current zkHost is instead of relying on Zk_HOST static setting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
