Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
I'm not sure it's necessary for backward compatibility since rolling restarts for config changes are not really an api the system provides. I'd think the ACL control and admin only API should be sufficient for security and would prefer to get rid of the flag. But if you must have it, we have to prevent both in memory config updates (most important) and config file updates if reconfig is disabled. This sounds like a small change in quorumpeer, but perhaps I'm forgetting something. Cheers Alex On Thu, Jun 22, 2017 at 11:06 PM Michael Han wrote: > Hi Alex, thanks for clarification! > > It makes sense to me that users should use reconfig instead of rolling > upgrade moving forward. The only concern is backward compatibility now we > drop the old rolling upgrade support: since 3.5.x needs to be backward > compatible with 3.4.x [1], I think we probably need support rolling upgrade > for 3.5 branch. > > As for this flag - I believe it's there and set to false because reconfig > is a security sensitive feature and for such features user has to opt in > explicitly. Our security team here also has similar recommendations when I > talked with them about what this feature could do. There are also some > discussions around this flag / why it's there in ZOOKEEPER-2014. > > [1] > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement > > > On Thu, Jun 22, 2017 at 10:39 PM, Alexander Shraer > wrote: > > > Hi Michael, > > > > The described behavior is the intended one - in 3.5 configuration is part > > of the synced state and is updated > > when the server syncs with the leader. The only rolling upgrade I tested > > was to upgrade the software version > > of the servers - this should still work. But I didn't try to support > > rolling upgrade for upgrading the configuration, > > since this should be done through reconfig. > > > > I'm still not sure what's the purpose of this flag btw. Why would someone > > want to do rolling restarts which are prone > > to inconsistencies and data loss, when they can use reconfig ? > > > > Alex > > > > > > > > > > On Thu, Jun 22, 2017 at 10:18 PM, Michael Han wrote: > > > > > reconfigEnabled only disables reconfig command when > > reconfigEnabled=false; > > > it does not disable the feature by mute all code paths of the reconfig > > > feature introduced in ZOOKEEPER-107. So regardless of the value of > > > reconfigEnabled, > > > 3.5.x ZK will create static config file and dynamic config file in any > > > cases. > > > > > > This might create a problem for users who want to do rolling upgrade > the > > > old way - because now the critical config information is not stored in > > > zoo.cfg anymore and modifying cfg.dynamic file manually will not work > > > because a reconfig needs to go through quorum processors. I think this > is > > > the problem described in the thread. > > > > > > Alex, is reconfig compatible with rolling upgrade? I don't find > anything > > > mentioned in ZOOKEEPER-107 about this. Currently I think the answer is > > no, > > > which means for 3.5.x the only way to change membership of cluster is > > > through reconfig. Could you confirm this conclusion? If that is the > case > > we > > > need patch the reconfigEnabled so it completely disable all code path > of > > > the reconfig feature to leave the static zoo.cfg intact. > > > > > > > > > On Thu, Jun 22, 2017 at 9:35 PM, Alexander Shraer > > > wrote: > > > > > > > This sounds like a bug in the implementation of reconfigEnabled. > > > > Could you please open a JIRA with the description you provided ? > > > > > > > > Out of curiosity, why do you disable reconfig ? It is intended > exactly > > > > to perform the changes you're trying to make, in a simple and correct > > > way. > > > > > > > > Thanks, > > > > Alex > > > > > > > > On Thu, Jun 22, 2017 at 3:17 PM, Guillermo Vega-Toro < > > > gvega...@us.ibm.com> > > > > wrote: > > > > > > > > > I'm still unable to make configuration changes when > > > reconfigEnabled=false > > > > > by updating zoo.cfg and restarting the servers. > > > > > > > > > > For example, I want to change the weight of one of my servers. I > edit > > > > > zoo.cfg on the server I want to change, and specify the group, > > > server.x, > > > > > and weight.x properties for all servers. I also remove the > > > > > dynamicConfigFile property and delete the dynamic config file. I > then > > > > > restart the server. As soon as the server starts, the dynamic > config > > > file > > > > > re-appears, and it has the last configuration, as if the changes I > > made > > > > in > > > > > zoo.cfg were ignored. The dynamic configuration file on the other > > > servers > > > > > also stays the same. > > > > > > > > > > What would be the correct way to achieve this (change a server's > > > weight, > > > > > or role) when reconfigEnabled=false and the CLI reconfig command > > cannot > > > > be > > > > > used? > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > -- > >
Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
Hi Alex, thanks for clarification! It makes sense to me that users should use reconfig instead of rolling upgrade moving forward. The only concern is backward compatibility now we drop the old rolling upgrade support: since 3.5.x needs to be backward compatible with 3.4.x [1], I think we probably need support rolling upgrade for 3.5 branch. As for this flag - I believe it's there and set to false because reconfig is a security sensitive feature and for such features user has to opt in explicitly. Our security team here also has similar recommendations when I talked with them about what this feature could do. There are also some discussions around this flag / why it's there in ZOOKEEPER-2014. [1] https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement On Thu, Jun 22, 2017 at 10:39 PM, Alexander Shraer wrote: > Hi Michael, > > The described behavior is the intended one - in 3.5 configuration is part > of the synced state and is updated > when the server syncs with the leader. The only rolling upgrade I tested > was to upgrade the software version > of the servers - this should still work. But I didn't try to support > rolling upgrade for upgrading the configuration, > since this should be done through reconfig. > > I'm still not sure what's the purpose of this flag btw. Why would someone > want to do rolling restarts which are prone > to inconsistencies and data loss, when they can use reconfig ? > > Alex > > > > > On Thu, Jun 22, 2017 at 10:18 PM, Michael Han wrote: > > > reconfigEnabled only disables reconfig command when > reconfigEnabled=false; > > it does not disable the feature by mute all code paths of the reconfig > > feature introduced in ZOOKEEPER-107. So regardless of the value of > > reconfigEnabled, > > 3.5.x ZK will create static config file and dynamic config file in any > > cases. > > > > This might create a problem for users who want to do rolling upgrade the > > old way - because now the critical config information is not stored in > > zoo.cfg anymore and modifying cfg.dynamic file manually will not work > > because a reconfig needs to go through quorum processors. I think this is > > the problem described in the thread. > > > > Alex, is reconfig compatible with rolling upgrade? I don't find anything > > mentioned in ZOOKEEPER-107 about this. Currently I think the answer is > no, > > which means for 3.5.x the only way to change membership of cluster is > > through reconfig. Could you confirm this conclusion? If that is the case > we > > need patch the reconfigEnabled so it completely disable all code path of > > the reconfig feature to leave the static zoo.cfg intact. > > > > > > On Thu, Jun 22, 2017 at 9:35 PM, Alexander Shraer > > wrote: > > > > > This sounds like a bug in the implementation of reconfigEnabled. > > > Could you please open a JIRA with the description you provided ? > > > > > > Out of curiosity, why do you disable reconfig ? It is intended exactly > > > to perform the changes you're trying to make, in a simple and correct > > way. > > > > > > Thanks, > > > Alex > > > > > > On Thu, Jun 22, 2017 at 3:17 PM, Guillermo Vega-Toro < > > gvega...@us.ibm.com> > > > wrote: > > > > > > > I'm still unable to make configuration changes when > > reconfigEnabled=false > > > > by updating zoo.cfg and restarting the servers. > > > > > > > > For example, I want to change the weight of one of my servers. I edit > > > > zoo.cfg on the server I want to change, and specify the group, > > server.x, > > > > and weight.x properties for all servers. I also remove the > > > > dynamicConfigFile property and delete the dynamic config file. I then > > > > restart the server. As soon as the server starts, the dynamic config > > file > > > > re-appears, and it has the last configuration, as if the changes I > made > > > in > > > > zoo.cfg were ignored. The dynamic configuration file on the other > > servers > > > > also stays the same. > > > > > > > > What would be the correct way to achieve this (change a server's > > weight, > > > > or role) when reconfigEnabled=false and the CLI reconfig command > cannot > > > be > > > > used? > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > -- > > Cheers > > Michael. > > > -- Cheers Michael.
Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
Hi Michael, The described behavior is the intended one - in 3.5 configuration is part of the synced state and is updated when the server syncs with the leader. The only rolling upgrade I tested was to upgrade the software version of the servers - this should still work. But I didn't try to support rolling upgrade for upgrading the configuration, since this should be done through reconfig. I'm still not sure what's the purpose of this flag btw. Why would someone want to do rolling restarts which are prone to inconsistencies and data loss, when they can use reconfig ? Alex On Thu, Jun 22, 2017 at 10:18 PM, Michael Han wrote: > reconfigEnabled only disables reconfig command when reconfigEnabled=false; > it does not disable the feature by mute all code paths of the reconfig > feature introduced in ZOOKEEPER-107. So regardless of the value of > reconfigEnabled, > 3.5.x ZK will create static config file and dynamic config file in any > cases. > > This might create a problem for users who want to do rolling upgrade the > old way - because now the critical config information is not stored in > zoo.cfg anymore and modifying cfg.dynamic file manually will not work > because a reconfig needs to go through quorum processors. I think this is > the problem described in the thread. > > Alex, is reconfig compatible with rolling upgrade? I don't find anything > mentioned in ZOOKEEPER-107 about this. Currently I think the answer is no, > which means for 3.5.x the only way to change membership of cluster is > through reconfig. Could you confirm this conclusion? If that is the case we > need patch the reconfigEnabled so it completely disable all code path of > the reconfig feature to leave the static zoo.cfg intact. > > > On Thu, Jun 22, 2017 at 9:35 PM, Alexander Shraer > wrote: > > > This sounds like a bug in the implementation of reconfigEnabled. > > Could you please open a JIRA with the description you provided ? > > > > Out of curiosity, why do you disable reconfig ? It is intended exactly > > to perform the changes you're trying to make, in a simple and correct > way. > > > > Thanks, > > Alex > > > > On Thu, Jun 22, 2017 at 3:17 PM, Guillermo Vega-Toro < > gvega...@us.ibm.com> > > wrote: > > > > > I'm still unable to make configuration changes when > reconfigEnabled=false > > > by updating zoo.cfg and restarting the servers. > > > > > > For example, I want to change the weight of one of my servers. I edit > > > zoo.cfg on the server I want to change, and specify the group, > server.x, > > > and weight.x properties for all servers. I also remove the > > > dynamicConfigFile property and delete the dynamic config file. I then > > > restart the server. As soon as the server starts, the dynamic config > file > > > re-appears, and it has the last configuration, as if the changes I made > > in > > > zoo.cfg were ignored. The dynamic configuration file on the other > servers > > > also stays the same. > > > > > > What would be the correct way to achieve this (change a server's > weight, > > > or role) when reconfigEnabled=false and the CLI reconfig command cannot > > be > > > used? > > > > > > Thanks > > > > > > > > > > > > -- > Cheers > Michael. >
Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
reconfigEnabled only disables reconfig command when reconfigEnabled=false; it does not disable the feature by mute all code paths of the reconfig feature introduced in ZOOKEEPER-107. So regardless of the value of reconfigEnabled, 3.5.x ZK will create static config file and dynamic config file in any cases. This might create a problem for users who want to do rolling upgrade the old way - because now the critical config information is not stored in zoo.cfg anymore and modifying cfg.dynamic file manually will not work because a reconfig needs to go through quorum processors. I think this is the problem described in the thread. Alex, is reconfig compatible with rolling upgrade? I don't find anything mentioned in ZOOKEEPER-107 about this. Currently I think the answer is no, which means for 3.5.x the only way to change membership of cluster is through reconfig. Could you confirm this conclusion? If that is the case we need patch the reconfigEnabled so it completely disable all code path of the reconfig feature to leave the static zoo.cfg intact. On Thu, Jun 22, 2017 at 9:35 PM, Alexander Shraer wrote: > This sounds like a bug in the implementation of reconfigEnabled. > Could you please open a JIRA with the description you provided ? > > Out of curiosity, why do you disable reconfig ? It is intended exactly > to perform the changes you're trying to make, in a simple and correct way. > > Thanks, > Alex > > On Thu, Jun 22, 2017 at 3:17 PM, Guillermo Vega-Toro > wrote: > > > I'm still unable to make configuration changes when reconfigEnabled=false > > by updating zoo.cfg and restarting the servers. > > > > For example, I want to change the weight of one of my servers. I edit > > zoo.cfg on the server I want to change, and specify the group, server.x, > > and weight.x properties for all servers. I also remove the > > dynamicConfigFile property and delete the dynamic config file. I then > > restart the server. As soon as the server starts, the dynamic config file > > re-appears, and it has the last configuration, as if the changes I made > in > > zoo.cfg were ignored. The dynamic configuration file on the other servers > > also stays the same. > > > > What would be the correct way to achieve this (change a server's weight, > > or role) when reconfigEnabled=false and the CLI reconfig command cannot > be > > used? > > > > Thanks > > > > > -- Cheers Michael.
Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
This sounds like a bug in the implementation of reconfigEnabled. Could you please open a JIRA with the description you provided ? Out of curiosity, why do you disable reconfig ? It is intended exactly to perform the changes you're trying to make, in a simple and correct way. Thanks, Alex On Thu, Jun 22, 2017 at 3:17 PM, Guillermo Vega-Toro wrote: > I'm still unable to make configuration changes when reconfigEnabled=false > by updating zoo.cfg and restarting the servers. > > For example, I want to change the weight of one of my servers. I edit > zoo.cfg on the server I want to change, and specify the group, server.x, > and weight.x properties for all servers. I also remove the > dynamicConfigFile property and delete the dynamic config file. I then > restart the server. As soon as the server starts, the dynamic config file > re-appears, and it has the last configuration, as if the changes I made in > zoo.cfg were ignored. The dynamic configuration file on the other servers > also stays the same. > > What would be the correct way to achieve this (change a server's weight, > or role) when reconfigEnabled=false and the CLI reconfig command cannot be > used? > > Thanks > >
RE: Version parameter passed to ZooKeeper.setACL
Done. https://issues.apache.org/jira/browse/ZOOKEEPER-2818 Thanks. --Brahma Reddy Battula -Original Message- From: Rakesh Radhakrishnan [mailto:rake...@apache.org] Sent: 23 June 2017 11:10 To: user@zookeeper.apache.org Subject: Re: Version parameter passed to ZooKeeper.setACL Agreed. Please feel free to raise a jira task for improving zkcli#setACL() javadoc. Rakesh On Fri, Jun 23, 2017 at 8:32 AM, Brahma Reddy Battula < brahmareddy.batt...@huawei.com> wrote: > One suggestion here. > > Java doc or argument can be improved which might not mislead..? > > i) public Stat setACL(final String path, List acl, int aversion > or > aclVersion) > ii) "should pass aclversion only"..something like this.. > > > > --Brahma Reddy Battula > > -Original Message- > From: Rakesh Radhakrishnan [mailto:rake...@apache.org] > Sent: 23 June 2017 10:47 > To: user@zookeeper.apache.org > Subject: Re: Version parameter passed to ZooKeeper.setACL > > Hi Arpit, > > Stat#aversion represents "the number of changes to the ACL of this znode." > On calling the zkcli#setACL api, internally ZK server will increase > the 'aversion' by one. If the given 'aversion' does not match the > znode's aversion it will throw BadVersionException. > > While invoking the #setACL api, you should pass "Stat.aversion". > > Rakesh > > On Thu, Jun 22, 2017 at 11:34 PM, Arpit Agarwal > > wrote: > > > Greetings, > > > > For the ZooKeeper.setACL call: > > > > https://github.com/apache/zookeeper/blob/master/src/java/ > > main/org/apache/zookeeper/ZooKeeper.java#L2368 > > > > public Stat setACL(final String path, List acl, int > > version) > > > > Should version be set to Stat.version or Stat.aversion? > > > > Thanks, > > Arpit > > > > > > >
Re: Version parameter passed to ZooKeeper.setACL
Agreed. Please feel free to raise a jira task for improving zkcli#setACL() javadoc. Rakesh On Fri, Jun 23, 2017 at 8:32 AM, Brahma Reddy Battula < brahmareddy.batt...@huawei.com> wrote: > One suggestion here. > > Java doc or argument can be improved which might not mislead..? > > i) public Stat setACL(final String path, List acl, int aversion or > aclVersion) > ii) "should pass aclversion only"..something like this.. > > > > --Brahma Reddy Battula > > -Original Message- > From: Rakesh Radhakrishnan [mailto:rake...@apache.org] > Sent: 23 June 2017 10:47 > To: user@zookeeper.apache.org > Subject: Re: Version parameter passed to ZooKeeper.setACL > > Hi Arpit, > > Stat#aversion represents "the number of changes to the ACL of this znode." > On calling the zkcli#setACL api, internally ZK server will increase the > 'aversion' by one. If the given 'aversion' does not match the znode's > aversion it will throw BadVersionException. > > While invoking the #setACL api, you should pass "Stat.aversion". > > Rakesh > > On Thu, Jun 22, 2017 at 11:34 PM, Arpit Agarwal > wrote: > > > Greetings, > > > > For the ZooKeeper.setACL call: > > > > https://github.com/apache/zookeeper/blob/master/src/java/ > > main/org/apache/zookeeper/ZooKeeper.java#L2368 > > > > public Stat setACL(final String path, List acl, int version) > > > > Should version be set to Stat.version or Stat.aversion? > > > > Thanks, > > Arpit > > > > > > >
RE: Version parameter passed to ZooKeeper.setACL
One suggestion here. Java doc or argument can be improved which might not mislead..? i) public Stat setACL(final String path, List acl, int aversion or aclVersion) ii) "should pass aclversion only"..something like this.. --Brahma Reddy Battula -Original Message- From: Rakesh Radhakrishnan [mailto:rake...@apache.org] Sent: 23 June 2017 10:47 To: user@zookeeper.apache.org Subject: Re: Version parameter passed to ZooKeeper.setACL Hi Arpit, Stat#aversion represents "the number of changes to the ACL of this znode." On calling the zkcli#setACL api, internally ZK server will increase the 'aversion' by one. If the given 'aversion' does not match the znode's aversion it will throw BadVersionException. While invoking the #setACL api, you should pass "Stat.aversion". Rakesh On Thu, Jun 22, 2017 at 11:34 PM, Arpit Agarwal wrote: > Greetings, > > For the ZooKeeper.setACL call: > > https://github.com/apache/zookeeper/blob/master/src/java/ > main/org/apache/zookeeper/ZooKeeper.java#L2368 > > public Stat setACL(final String path, List acl, int version) > > Should version be set to Stat.version or Stat.aversion? > > Thanks, > Arpit > > >
Re: Version parameter passed to ZooKeeper.setACL
Hi Arpit, Stat#aversion represents "the number of changes to the ACL of this znode." On calling the zkcli#setACL api, internally ZK server will increase the 'aversion' by one. If the given 'aversion' does not match the znode's aversion it will throw BadVersionException. While invoking the #setACL api, you should pass "Stat.aversion". Rakesh On Thu, Jun 22, 2017 at 11:34 PM, Arpit Agarwal wrote: > Greetings, > > For the ZooKeeper.setACL call: > > https://github.com/apache/zookeeper/blob/master/src/java/ > main/org/apache/zookeeper/ZooKeeper.java#L2368 > > public Stat setACL(final String path, List acl, int version) > > Should version be set to Stat.version or Stat.aversion? > > Thanks, > Arpit > > >
Re: Version parameter passed to ZooKeeper.setACL
Stat.aversion > On Jun 22, 2017, at 1:04 PM, Arpit Agarwal wrote: > > Greetings, > > For the ZooKeeper.setACL call: > > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/ZooKeeper.java#L2368 > > public Stat setACL(final String path, List acl, int version) > > Should version be set to Stat.version or Stat.aversion? > > Thanks, > Arpit > >
Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false
I'm still unable to make configuration changes when reconfigEnabled=false by updating zoo.cfg and restarting the servers. For example, I want to change the weight of one of my servers. I edit zoo.cfg on the server I want to change, and specify the group, server.x, and weight.x properties for all servers. I also remove the dynamicConfigFile property and delete the dynamic config file. I then restart the server. As soon as the server starts, the dynamic config file re-appears, and it has the last configuration, as if the changes I made in zoo.cfg were ignored. The dynamic configuration file on the other servers also stays the same. What would be the correct way to achieve this (change a server's weight, or role) when reconfigEnabled=false and the CLI reconfig command cannot be used? Thanks
Version parameter passed to ZooKeeper.setACL
Greetings, For the ZooKeeper.setACL call: https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/ZooKeeper.java#L2368 public Stat setACL(final String path, List acl, int version) Should version be set to Stat.version or Stat.aversion? Thanks, Arpit
