Re: ZooKeeper & dynamic discovery (apache river)
Jonathan Reichhold wrote: I was just expressing an opinion as someone who watches the list. I'm not trying to drive you toward any decision, but have watched how River has stayed in incubation for 2 years. I've used JINI on projects and was mostly happy, but the bugs in it were never resolved last I looked. I was attempting to ask what the policy of Zookeeper was in regards to something that hasn't gotten out of incubation. Sorry for the slow response but I wanted to make sure I understood the issues before commenting. Jonathan, I do appreciate you bringing up this issue, it's not something I had considered (incubation), although I did look at River and it's list activity before initially commenting. According to the response that I got from the incubator community it's fine for TLPs to use incubator code, in particular this snippet: I don't think this is a problem. Incubator releases are approved by the Incubator PMC, and have had as much (or more) legal review as releases by other TLPs. We would want the submitted code to depend on a released version of River, not code that's unreleased (not a river jar built from svn for example) full details: http://bit.ly/bbqk4D That said, I did have some concern regarding River's apparent lack of progress. Mature projects might have very infrequent releases (look at log4j, or Forrest), but it did seem a bit unusual that a community in incubation would have so infrequent a release cycle, at the very least one would expect fix releases! This was why I suggested submitting as a contrib. The functionality seems useful to our user base and having it available in contrib would be a good thing IMO. Please don't take my comments as a statement on this project and what you should do. I have no commit privileges or management of the project, and just wanted to ask the question. I believe your comments were both justified and useful, keep them coming. :-) Brian, not sure where you guys are with your thinking on this. Feel free to open a JIRA, collect further feedback, and submit a patch. Regards, Patrick Jonathan On Sun, Mar 21, 2010 at 9:38 AM, Brian Murphy wrote: On Sun, Mar 21, 2010 at 12:37 AM, Jonathan Reichhold < jonathan.reichh...@gmail.com> wrote: Apache River is dying from lack of updates. Hmm, I suppose "dying" is a matter of opinion, or one's perspective. If you're talking about the lack of an official release, no argument there. But, for what it's worth, that might be more a function of the many differing opinions being voiced by the various parties interested in that project; which has driven some away, but has been viewed by others as healthy discourse. For example, because the river codebase provides an infrastructure rather than a specific application, and is fairly mature and stable, some of the river meritocracy feel that the project should move more slowly than application based apache projects typically move, whereas others feel just the opposite. I generally leave these sort of arguments for others to worry about though. I'm usually more interested in whether the code serves the needs of the project I'm on (which in this case, both river and zookeeper do). Fortunately, the project I'm currently on doesn't need any major new features from river. The numerous patches and updates that have been feeding into the pending 2.1.1 release (currently under vote) have been more than enough to serve our needs, and have been put to quite good use. But of course, your needs and experiences may be different. not sure tying to a project which hasn't moved since 2008 is a good idea for a contribution to Zookeeeper. Okay, thanks for the honest answer, Jonathan. This is what we were trying to find out by posing the original question to the Zookeeper community. Since we don't want to be disruptive, we'll simply continue developing the code in our own namespace. No harm, no foul. Thanks, Brian
Re: ZooKeeper & dynamic discovery (apache river)
I was just expressing an opinion as someone who watches the list. I'm not trying to drive you toward any decision, but have watched how River has stayed in incubation for 2 years. I've used JINI on projects and was mostly happy, but the bugs in it were never resolved last I looked. I was attempting to ask what the policy of Zookeeper was in regards to something that hasn't gotten out of incubation. Please don't take my comments as a statement on this project and what you should do. I have no commit privileges or management of the project, and just wanted to ask the question. Jonathan On Sun, Mar 21, 2010 at 9:38 AM, Brian Murphy wrote: > On Sun, Mar 21, 2010 at 12:37 AM, Jonathan Reichhold < > jonathan.reichh...@gmail.com> wrote: > > Apache River is dying from lack of updates. > > > Hmm, I suppose "dying" is a matter of opinion, or one's > perspective. > > If you're talking about the lack of an official release, > no argument there. But, for what it's worth, that might > be more a function of the many differing opinions > being voiced by the various parties interested in that > project; which has driven some away, but has been > viewed by others as healthy discourse. > > For example, because the river codebase provides > an infrastructure rather than a specific application, and is > fairly mature and stable, some of the river meritocracy feel > that the project should move more slowly than application > based apache projects typically move, whereas others feel > just the opposite. I generally leave these sort of arguments > for others to worry about though. I'm usually more interested > in whether the code serves the needs of the project I'm on > (which in this case, both river and zookeeper do). > > Fortunately, the project I'm currently on doesn't need any > major new features from river. The numerous patches > and updates that have been feeding into the pending 2.1.1 > release (currently under vote) have been more than enough > to serve our needs, and have been put to quite good use. > But of course, your needs and experiences may be different. > > > > not sure tying to a project which hasn't moved since 2008 is a > > good idea for a contribution to Zookeeeper. > > > Okay, thanks for the honest answer, Jonathan. This is what > we were trying to find out by posing the original question to > the Zookeeper community. Since we don't want to be disruptive, > we'll simply continue developing the code in our own namespace. > No harm, no foul. > > Thanks, > Brian >
Re: ZooKeeper & dynamic discovery (apache river)
However, as I understand it, this would involve the development of a contrib module, not a set of patches to trunk. Therefore there's no issue of 'tying' ZooKeeper to River. I'm ill placed to judge whether this would be a contribution that many people would use, but we should not reject it before we even discuss it on a JIRA. It certainly seems like a positive contribution, and we have historically been interested in integrating with service frameworks. Brian, I'd encourage you to open a JIRA and contribute your code. We'll discuss the technical merits there. cheers, Henry On 21 March 2010 09:38, Brian Murphy wrote: > On Sun, Mar 21, 2010 at 12:37 AM, Jonathan Reichhold < > jonathan.reichh...@gmail.com> wrote: > > Apache River is dying from lack of updates. > > > Hmm, I suppose "dying" is a matter of opinion, or one's > perspective. > > If you're talking about the lack of an official release, > no argument there. But, for what it's worth, that might > be more a function of the many differing opinions > being voiced by the various parties interested in that > project; which has driven some away, but has been > viewed by others as healthy discourse. > > For example, because the river codebase provides > an infrastructure rather than a specific application, and is > fairly mature and stable, some of the river meritocracy feel > that the project should move more slowly than application > based apache projects typically move, whereas others feel > just the opposite. I generally leave these sort of arguments > for others to worry about though. I'm usually more interested > in whether the code serves the needs of the project I'm on > (which in this case, both river and zookeeper do). > > Fortunately, the project I'm currently on doesn't need any > major new features from river. The numerous patches > and updates that have been feeding into the pending 2.1.1 > release (currently under vote) have been more than enough > to serve our needs, and have been put to quite good use. > But of course, your needs and experiences may be different. > > > > not sure tying to a project which hasn't moved since 2008 is a > > good idea for a contribution to Zookeeeper. > > > Okay, thanks for the honest answer, Jonathan. This is what > we were trying to find out by posing the original question to > the Zookeeper community. Since we don't want to be disruptive, > we'll simply continue developing the code in our own namespace. > No harm, no foul. > > Thanks, > Brian > -- Henry Robinson Software Engineer Cloudera 415-994-6679
Re: ZooKeeper & dynamic discovery (apache river)
On Sun, Mar 21, 2010 at 12:37 AM, Jonathan Reichhold < jonathan.reichh...@gmail.com> wrote: Apache River is dying from lack of updates. Hmm, I suppose "dying" is a matter of opinion, or one's perspective. If you're talking about the lack of an official release, no argument there. But, for what it's worth, that might be more a function of the many differing opinions being voiced by the various parties interested in that project; which has driven some away, but has been viewed by others as healthy discourse. For example, because the river codebase provides an infrastructure rather than a specific application, and is fairly mature and stable, some of the river meritocracy feel that the project should move more slowly than application based apache projects typically move, whereas others feel just the opposite. I generally leave these sort of arguments for others to worry about though. I'm usually more interested in whether the code serves the needs of the project I'm on (which in this case, both river and zookeeper do). Fortunately, the project I'm currently on doesn't need any major new features from river. The numerous patches and updates that have been feeding into the pending 2.1.1 release (currently under vote) have been more than enough to serve our needs, and have been put to quite good use. But of course, your needs and experiences may be different. > not sure tying to a project which hasn't moved since 2008 is a > good idea for a contribution to Zookeeeper. Okay, thanks for the honest answer, Jonathan. This is what we were trying to find out by posing the original question to the Zookeeper community. Since we don't want to be disruptive, we'll simply continue developing the code in our own namespace. No harm, no foul. Thanks, Brian
Re: ZooKeeper & dynamic discovery (apache river)
Apache River is dying from lack of updates. While JINI seems like a good idea here, not sure tying to a project which hasn't moved since 2008 is a good idea for a contribution to Zookeeeper. Would love to hear otherwise, but this has been my experieince with JINI and River. Jonathan On Fri, Mar 19, 2010 at 3:52 PM, Patrick Hunt wrote: > Hi Brian, sounds like a cool project. While I'm not super familiar with > Apache River or Jini I do think what you are describing could be beneficial. > I saw the following recently, is what you are describing a possible solution > for this situation? http://bit.ly/dabIev > > Historically specifying the ZK server host/port list for client > applications (zookeeper client sessions) has been an issue for some users. > If we could provide an option that used River to discover this I believe it > would be useful. > > We'd be happy to work with you to get something into our contrib module. We > recently did ivy integration for BookKeeper contrib so there's prior art > here, we just need to duplicate and change the dependencies. I'd help you > with this. > > Feel free to open a JIRA and submit a patch to get started: > https://issues.apache.org/jira/browse/ZOOKEEPER > http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute > > Regards, > > Patrick > > > Brian Murphy wrote: > >> I've recently been assigned to a project using >> ZooKeeper and so I'm just a (new) user of ZooKeeper, >> rather than someone interested in making changes to >> anything internal to ZooKeeper. With that in mind, I'm not >> sure if this is the right list for the question I have. If it's not, >> then please forgive the interruption; and I hope someone >> will (gently) point me to the correct list to pose the >> question. >> >> So here's the context to the question I'd like to ask: >> >> The project I've been assigned to would like to be able >> to do the following with respect to ZooKeeper: >> >> - dynamically discover each peer in an ensemble >> - as part of the dynamic discovery of a given peer, >> advertise the peer's configuration; in particular, the >> clientPort, and the peer and election ports and >> peer address >> - administratively shutdown (in a graceful fashion) a >> given peer >> >> To achieve this we've wrapped QuorumPeerMain >> and ZooKeeperServerMain in an apache river >> based backend impl class that registers a frontend >> proxy with a river service directory, from which that >> proxy (and associated configuration info attribute) >> can be dynamically discovered. >> >> None of the work above requires any changes to >> the existing ZooKeeper codebase; merely the addition >> of a handful of classes that can be used or ignored. >> >> The question I have then, is whether anyone in the >> ZooKeeper community would be interested in this work >> being contributed. If not, no problem. Since the classes >> above currently do what we want, we'll simply move >> those new classes into our project's namespace, link >> to ZooKeeper in the normal way, and go from there. >> >> So before we spend any time on trying to figure out how >> to use ivy to retrieve the appropriate river jars (which we >> may need help with) or creating example apps & configs >> to help others to understand and try out this functionality, >> we thought we'd first try to determine if there's even any >> interest; and, if there is, then engage the community in >> how to proceed. >> >> Anyway, if there's any interest, please let us know. >> >> Regards, >> Brian >> >>
Re: ZooKeeper & dynamic discovery (apache river)
Hi Brian, sounds like a cool project. While I'm not super familiar with Apache River or Jini I do think what you are describing could be beneficial. I saw the following recently, is what you are describing a possible solution for this situation? http://bit.ly/dabIev Historically specifying the ZK server host/port list for client applications (zookeeper client sessions) has been an issue for some users. If we could provide an option that used River to discover this I believe it would be useful. We'd be happy to work with you to get something into our contrib module. We recently did ivy integration for BookKeeper contrib so there's prior art here, we just need to duplicate and change the dependencies. I'd help you with this. Feel free to open a JIRA and submit a patch to get started: https://issues.apache.org/jira/browse/ZOOKEEPER http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute Regards, Patrick Brian Murphy wrote: I've recently been assigned to a project using ZooKeeper and so I'm just a (new) user of ZooKeeper, rather than someone interested in making changes to anything internal to ZooKeeper. With that in mind, I'm not sure if this is the right list for the question I have. If it's not, then please forgive the interruption; and I hope someone will (gently) point me to the correct list to pose the question. So here's the context to the question I'd like to ask: The project I've been assigned to would like to be able to do the following with respect to ZooKeeper: - dynamically discover each peer in an ensemble - as part of the dynamic discovery of a given peer, advertise the peer's configuration; in particular, the clientPort, and the peer and election ports and peer address - administratively shutdown (in a graceful fashion) a given peer To achieve this we've wrapped QuorumPeerMain and ZooKeeperServerMain in an apache river based backend impl class that registers a frontend proxy with a river service directory, from which that proxy (and associated configuration info attribute) can be dynamically discovered. None of the work above requires any changes to the existing ZooKeeper codebase; merely the addition of a handful of classes that can be used or ignored. The question I have then, is whether anyone in the ZooKeeper community would be interested in this work being contributed. If not, no problem. Since the classes above currently do what we want, we'll simply move those new classes into our project's namespace, link to ZooKeeper in the normal way, and go from there. So before we spend any time on trying to figure out how to use ivy to retrieve the appropriate river jars (which we may need help with) or creating example apps & configs to help others to understand and try out this functionality, we thought we'd first try to determine if there's even any interest; and, if there is, then engage the community in how to proceed. Anyway, if there's any interest, please let us know. Regards, Brian
ZooKeeper & dynamic discovery (apache river)
I've recently been assigned to a project using ZooKeeper and so I'm just a (new) user of ZooKeeper, rather than someone interested in making changes to anything internal to ZooKeeper. With that in mind, I'm not sure if this is the right list for the question I have. If it's not, then please forgive the interruption; and I hope someone will (gently) point me to the correct list to pose the question. So here's the context to the question I'd like to ask: The project I've been assigned to would like to be able to do the following with respect to ZooKeeper: - dynamically discover each peer in an ensemble - as part of the dynamic discovery of a given peer, advertise the peer's configuration; in particular, the clientPort, and the peer and election ports and peer address - administratively shutdown (in a graceful fashion) a given peer To achieve this we've wrapped QuorumPeerMain and ZooKeeperServerMain in an apache river based backend impl class that registers a frontend proxy with a river service directory, from which that proxy (and associated configuration info attribute) can be dynamically discovered. None of the work above requires any changes to the existing ZooKeeper codebase; merely the addition of a handful of classes that can be used or ignored. The question I have then, is whether anyone in the ZooKeeper community would be interested in this work being contributed. If not, no problem. Since the classes above currently do what we want, we'll simply move those new classes into our project's namespace, link to ZooKeeper in the normal way, and go from there. So before we spend any time on trying to figure out how to use ivy to retrieve the appropriate river jars (which we may need help with) or creating example apps & configs to help others to understand and try out this functionality, we thought we'd first try to determine if there's even any interest; and, if there is, then engage the community in how to proceed. Anyway, if there's any interest, please let us know. Regards, Brian
