Re: [Openstack] cfg usage - option registration, global objects
On Wed, Jun 6, 2012 at 10:58 AM, Mark McLoughlin mar...@redhat.com wrote: On Tue, 2012-06-05 at 17:25 -0400, Mark Washenberger wrote: Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 12:21 -0400, Mark Washenberger wrote: http://wiki.openstack.org/CommonLibrary#Incubation Once an api is in incubation, if you make a change to it, you are expected to update all the other openstack projects (not just core projects?) to make them work with the new api. Am I understanding this requirement correctly? Yes, pretty much. I should clarify this - I don't think someone improving an API in openstack-common absolutely must update all projects that use it, but I think he/she often will update the major users to make sure the new API works. But the reality is that projects which use openstack-common need to be prepared that someday they might re-sync with latest openstack-common using update.py and find the API has changed. The alternative is that you don't make backwards incompatible API changes. ... What alternative strategy are you suggesting? That if glance, quantum, cinder and ceilometer want to re-use Nova's RPC code, they should copy-and-paste it and hack it to their needs? I don't think our only options here are immediate adoption and relative chaos. Here's what I would like to see: Quantum, cinder, and ceilometer get together, recognize a shared need for rpc, acknowledge the successes and failures of the nova.rpc library, and create a better implementation with eventual adoption by Nova kept in mind. Doesn't that sound better? This approach seems much nicer to me, because I believe that code reuse is likely to be detrimental unless the code we're talking about was created with reuse and generality in mind. Since I find that suggestion implausible regarding nova.rpc in particular, I think we will do better overall avoiding its wider adoption. Please, forgive me if I'm being drawn in falsely by the allure of better code. Code structure and quality *is* something I obsess about. But I appreciate the need to be practical in the short term as well, if I am not always the best at articulating it. The agreement from various quarters about the problems with the current nova.rpc gave me hope that we could craft a better rpc library without too much delay, even if I myself can only contribute in a small way to that effort. I think the summary is that you'd like (someone else?) to start from scratch on a new RPC API in openstack-common, whereas Russell has gone for the approach of taking Nova's RPC API and improving it iteratively. In the absence of someone appearing with that new idealised RPC API, I think it's reasonable for Russell to proceed with his approach. Yes, please, keep going! We're too close now to stop now. I do have some enhancements to propose, but I think I can make them in a backwards compatible way once the existing code is in the common lib. Ceilometer is currently importing bits we need either directly from nova or openstack-common (by importing I mean literally using the import statement in our code, not copying the required modules into the ceilometer code base). I was under the impression that this is how we wanted (new) projects to use openstack-common, but maybe I misunderstood? Should we be planning to copy code out of openstack-common into the ceilometer repository? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Wed, 2012-06-06 at 12:05 -0400, Doug Hellmann wrote: Ceilometer is currently importing bits we need either directly from nova or openstack-common (by importing I mean literally using the import statement in our code, not copying the required modules into the ceilometer code base). I was under the impression that this is how we wanted (new) projects to use openstack-common, but maybe I misunderstood? Should we be planning to copy code out of openstack-common into the ceilometer repository? Yep, the rpc code will be in incubation for a while: http://wiki.openstack.org/CommonLibrary Add an openstack-common.conf and use update.py to sync it across. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
But the reality is that projects which use openstack-common need to be prepared that someday they might re-sync with latest openstack-common using update.py and find the API has changed. That sounds good, especially during the early parts of incubation. In the absence of someone appearing with that new idealised RPC API, I think it's reasonable for Russell to proceed with his approach. Looking more closely at Russell's improvements, and thinking about the big changes I think we ought to make, it seems he's done a lot of the good work already. I would like for us to kick the global and C-style polymorphism habits in rpc, but I guess the best I can do is try to add these changes after its in common. Sorry if this has already been posted somewhere and I just can't find it, but is there an openstack common weekly meeting where you guys talk about your blueprints and determine what is going into common and in what form? I think I can be less disruptive if I'm involved in these discussions much earlier. Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 17:25 -0400, Mark Washenberger wrote: Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 12:21 -0400, Mark Washenberger wrote: http://wiki.openstack.org/CommonLibrary#Incubation Once an api is in incubation, if you make a change to it, you are expected to update all the other openstack projects (not just core projects?) to make them work with the new api. Am I understanding this requirement correctly? Yes, pretty much. I should clarify this - I don't think someone improving an API in openstack-common absolutely must update all projects that use it, but I think he/she often will update the major users to make sure the new API works. But the reality is that projects which use openstack-common need to be prepared that someday they might re-sync with latest openstack-common using update.py and find the API has changed. The alternative is that you don't make backwards incompatible API changes. ... What alternative strategy are you suggesting? That if glance, quantum, cinder and ceilometer want to re-use Nova's RPC code, they should copy-and-paste it and hack it to their needs? I don't think our only options here are immediate adoption and relative chaos. Here's what I would like to see: Quantum, cinder, and ceilometer get together, recognize a shared need for rpc, acknowledge the successes and failures of the nova.rpc library, and create a better implementation with eventual adoption by Nova kept in mind. Doesn't that sound better? This approach seems much nicer to me, because I believe that code reuse is likely to be detrimental unless the code we're talking about was created with reuse and generality in mind. Since I find that suggestion implausible regarding nova.rpc in particular, I think we will do better overall avoiding its wider adoption. Please, forgive me if I'm being drawn in falsely by the allure of better code. Code structure and quality *is* something I obsess about. But I appreciate the need to be practical in the short term as well, if I am not always the best at articulating it. The agreement from various quarters about the problems with the current nova.rpc gave me hope that we could craft a better rpc library without too much delay, even if I myself can only contribute in a small way to that effort. I think the summary is that you'd like (someone else?) to start from scratch on a new RPC API in openstack-common, whereas Russell has gone for the approach of taking Nova's RPC API and improving it iteratively. In the absence of someone appearing with that new idealised RPC API, I think it's reasonable for Russell to proceed with his approach. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Wed, 2012-06-06 at 15:41 -0400, Mark Washenberger wrote: Sorry if this has already been posted somewhere and I just can't find it, but is there an openstack common weekly meeting where you guys talk about your blueprints and determine what is going into common and in what form? I think I can be less disruptive if I'm involved in these discussions much earlier. We haven't been doing meetings, but there was a long thread on this a while back: https://lists.launchpad.net/openstack/msg09456.html Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Fri, 2012-06-01 at 10:37 -0400, Mark Washenberger wrote: Hi Mark, Please forgive the top-posting! I always get way too wordy with inline replies. Regarding configuration, I think there is another option I'd like us to adopt. We should implement the code as in your option #1, but then implement convenience factories that give the appearance of option #3 or #2, or both, you pick. From your examples it might look something like this: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) def connection_from_global_conf(): return Connection(CONF.broker_hostname, CONF.broker_port) I think its pretty necessary that we don't do option #3 directly. There are some important use cases to consider, like migrating from one rpc implementation to another where you might want an adapter that can relay messages from one to the other. Also, cells with kombu at least requires that one process be able to talk to multiple brokers. Yeah, I think that all makes sense. Depending on the context, we might want to require only one of #1, #2 or #3 to be supported in order for the API to be added to openstack-common initially. I think there'll be good reasons in different cases why only one of the options is required by initial users of the API. Regarding incubation, I suppose I am confused. At what point during incubation do other projects start to use the shared library? I would imagine the answer to be after incubation but it sounds like there are several projects very eager to adopt rpc as soon as it lands in openstack common, even before incubation is complete. If incubation happens before the calcifying effects of shared use set in, then it sounds like a great place to address the other rpc-specific concerns we've talked about. Otherwise I guess we're stuck where I thought we were, where the bar needs to be set pretty high to initially land in os-common. http://wiki.openstack.org/CommonLibrary#Incubation In openstack-common, Incubation - Core is we're ready to commit to backward compatibility. None of the APIs have reached that point yet. While an API is incubating, other projects can still use it and the API can make backwards incompatible changes. For example, I made a backwards incompatible change to the cfg.ConfigOpts() constructor recently and updated all projects to use the new API. That's exactly the case here - we want openstack.common.rpc to initially be a reasonable API that can be used by multiple OpenStack projects (say nova, glance and quantum) but that doesn't mean we need to commit to maintaining backwards compatibility for the API we initially add. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
+1 :-) d On Fri, Jun 1, 2012 at 10:37 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: Hi Mark, Please forgive the top-posting! I always get way too wordy with inline replies. Regarding configuration, I think there is another option I'd like us to adopt. We should implement the code as in your option #1, but then implement convenience factories that give the appearance of option #3 or #2, or both, you pick. From your examples it might look something like this: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) def connection_from_global_conf(): return Connection(CONF.broker_hostname, CONF.broker_port) I think its pretty necessary that we don't do option #3 directly. There are some important use cases to consider, like migrating from one rpc implementation to another where you might want an adapter that can relay messages from one to the other. Also, cells with kombu at least requires that one process be able to talk to multiple brokers. Regarding incubation, I suppose I am confused. At what point during incubation do other projects start to use the shared library? I would imagine the answer to be after incubation but it sounds like there are several projects very eager to adopt rpc as soon as it lands in openstack common, even before incubation is complete. If incubation happens before the calcifying effects of shared use set in, then it sounds like a great place to address the other rpc-specific concerns we've talked about. Otherwise I guess we're stuck where I thought we were, where the bar needs to be set pretty high to initially land in os-common. Mark McLoughlin mar...@redhat.com said: Hi Mark, On Thu, 2012-05-31 at 10:48 -0400, Mark Washenberger wrote: Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. This is a fairly general design question for openstack-common. If the behaviour of any given API should be dependent on the configuration of the service (i.e. in nova.conf, glance-api.conf, keystone.conf), then how should we design the API to handle that? Three options: 1) The API knows nothing about cfg: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 2) The API is passed a ConfigOpts instance: class Connection(object): def __init__(self, conf): self.cnx = self.connect(conf.broker_hostname, conf.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 3) The API uses the global config object class Connection(object): def __init__(self): self.cnx = self.connect(CONF.broker_hostname, CONF.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) I see (1) as having value if we want the API to be usable outside OpenStack. That might be worthwhile long term, but openstack-common's goal is to provide APIs shared between OpenStack projects. (2) and (3) have the huge benefit of configuration consistency - the same configuration keys, defaults, etc. shared across projects. (2) has value if there are cases where we want to use a non-global cfg object. Quantum might have this use case if it continues parsing plugin configuration separately from its main configuration. We'll see. (3) has value if all use cases are using a global object. I can see us getting to a point where we support both (2) and (3). I'm dubious of the value of (1) in the medium term, but maybe we might find a compelling use case for it. I'm also very happy to contribute modifications that would eliminate that connection. There are a few other reasons
Re: [Openstack] cfg usage - option registration, global objects
On Tue, 2012-06-05 at 12:48 -0400, Duncan McGreggor wrote: +1 :-) In all seriousness - Mark made two separate points. Which one are you top-posting a +1 to? On Fri, Jun 1, 2012 at 10:37 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: Hi Mark, Please forgive the top-posting! I always get way too wordy with inline replies. point 1 Regarding configuration, I think there is another option I'd like us to adopt. We should implement the code as in your option #1, but then implement convenience factories that give the appearance of option #3 or #2, or both, you pick. From your examples it might look something like this: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) def connection_from_global_conf(): return Connection(CONF.broker_hostname, CONF.broker_port) I think its pretty necessary that we don't do option #3 directly. There are some important use cases to consider, like migrating from one rpc implementation to another where you might want an adapter that can relay messages from one to the other. Also, cells with kombu at least requires that one process be able to talk to multiple brokers. /point 1 point 2 Regarding incubation, I suppose I am confused. At what point during incubation do other projects start to use the shared library? I would imagine the answer to be after incubation but it sounds like there are several projects very eager to adopt rpc as soon as it lands in openstack common, even before incubation is complete. If incubation happens before the calcifying effects of shared use set in, then it sounds like a great place to address the other rpc-specific concerns we've talked about. Otherwise I guess we're stuck where I thought we were, where the bar needs to be set pretty high to initially land in os-common. /point 2 Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On 06/05/2012 05:35 PM, Russell Bryant wrote: On 06/05/2012 05:25 PM, Mark Washenberger wrote: Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 12:21 -0400, Mark Washenberger wrote: http://wiki.openstack.org/CommonLibrary#Incubation Once an api is in incubation, if you make a change to it, you are expected to update all the other openstack projects (not just core projects?) to make them work with the new api. Am I understanding this requirement correctly? Yes, pretty much. The alternative is that you don't make backwards incompatible API changes. ... What alternative strategy are you suggesting? That if glance, quantum, cinder and ceilometer want to re-use Nova's RPC code, they should copy-and-paste it and hack it to their needs? I don't think our only options here are immediate adoption and relative chaos. Here's what I would like to see: Quantum, cinder, and ceilometer get together, recognize a shared need for rpc, acknowledge the successes and failures of the nova.rpc library, and create a better implementation with eventual adoption by Nova kept in mind. Doesn't that sound better? This approach seems much nicer to me, because I believe that code reuse is likely to be detrimental unless the code we're talking about was created with reuse and generality in mind. Since I find that suggestion implausible regarding nova.rpc in particular, I think we will do better overall avoiding its wider adoption. Please, forgive me if I'm being drawn in falsely by the allure of better code. Code structure and quality *is* something I obsess about. But I appreciate the need to be practical in the short term as well, if I am not always the best at articulating it. The agreement from various quarters about the problems with the current nova.rpc gave me hope that we could craft a better rpc library without too much delay, even if I myself can only contribute in a small way to that effort. That all sounds nice in theory, but like you alluded to here, who specifically is going to sign up to do that? I can finish the move of the current code (I have it ready to propose, actually), but I have other things I should work on after that. How about the stakeholders interested in using this code in other projects? Do you want the current code in now (with the likelihood of having to adapt to some API changes later), or would you like to get together and work on something new? If so, I'll just toss the proposal of the current code for common and let someone else drive the new thing in instead. By the way, the move of the current code is ready if we agree to proceed with it: https://review.openstack.org/#/c/8206/ -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On 06/01/2012 12:47 AM, Russell Bryant wrote: On 05/31/2012 04:21 PM, Mark McLoughlin wrote: I expect to do a pretty detailed review of the patch to add rpc to openstack-common and make some API design recommendations. But I think we'll have to balance some of those design ideas between stuff we really should fix before it goes into openstack-common vs stuff that would need to be fixed before the API comes out of incubation. I've been pushing the code with a goal that the submission to openstack-common is *only* a matter of changing the imports. Up to this point, it has mostly been a matter of removing dependencies on other parts of nova. We are actually at that point now ... unless there are other changes that are going to block it. I'd rather just work on improvements people would like to see while the code is in incubation in openstack-common. People are already copying it (in a feature branch for Quantum, Cinder, the Heat project, at least), so moving it as is will leave us in a better spot than we are now. I agree with Russell. On the Quantum side we are waiting for the code so that we can move ahead with a number of blueprints. I understand that there is also a solution for versioning. Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. I'm also very happy to contribute modifications that would eliminate that connection. There are a few other reasons I'm concerned about nova's rpc implementation becoming the de facto standard. It has grown up organically and as such has some significant issues we should probably address before we create a precedent that other projects must adopt it to be good community members. From a brief scan, it seems to me that a restructured implementation of rpc that lands in common should * not be tied up with eventlet on the consumer side * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) * not depend on a global singleton _RPC_IMPL * leave out/refactor some unused or non-general aspects, such as multicall, fanout_cast_to_server, notify, and fanout_cast (not so sure about that last one) I'm happy to work on some of these but I probably can't do the whole thing. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Thu, May 31, 2012 at 10:48 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. I'm also very happy to contribute modifications that would eliminate that connection. There are a few other reasons I'm concerned about nova's rpc implementation becoming the de facto standard. It has grown up organically and as such has some significant issues we should probably address before we create a precedent that other projects must adopt it to be good community members. From a brief scan, it seems to me that a restructured implementation of rpc that lands in common should * not be tied up with eventlet on the consumer side * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) * not depend on a global singleton _RPC_IMPL * leave out/refactor some unused or non-general aspects, such as multicall, fanout_cast_to_server, notify, and fanout_cast (not so sure about that last one) It would also be useful if it had a way to subscribe to notification messages. From what I can tell, notifications don't work with any of the consumers in the existing drivers because those consumers all want to dispatch to a method invocation, but notifications aren't structured appropriately for that. I would like to abstract the messaging layer out of the existing nova RPC library and then build a compatible RPC layer on top of it. Notifications and other simple messages (such as meter data for ceilometer) would use the lower layer, and communication that truly works like RPC could use the higher level API. We should be able to build an agnostic RPC layer that uses the messaging driver, instead of having the two concepts bound up together. I'm happy to work on some of these but I probably can't do the whole thing. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Thu, May 31, 2012, Mark Washenberger mark.washenber...@rackspace.com wrote: * not be tied up with eventlet on the consumer side Yes! * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) My XenAPI idempotency branch delays ACKs until after it's done processing the message to ensure we get the message again after restart. https://review.openstack.org/#/c/7323 I can't say that I'm happy with the changes I've made to safely support delayed ACKs. As far as implementations go, the Qpid docs say messages should be acknowledged, but I don't see anywhere in the code where that happens. I haven't dug too far into it to figure out this discrepancy. While the 0MQ code hasn't been merged yet, 0MQ appears to have no concept of acknowledging messages. Between the messy code in supporting delayed ACKs and the fact that some RPC implementations don't appear to support ACKs at all, it may mean reliability will need to be implemented elsewhere. The proposed orchestration work could possibly work as an alternative. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Thu, May 31, 2012 at 11:26 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Thu, May 31, 2012 at 10:48 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. I'm also very happy to contribute modifications that would eliminate that connection. There are a few other reasons I'm concerned about nova's rpc implementation becoming the de facto standard. It has grown up organically and as such has some significant issues we should probably address before we create a precedent that other projects must adopt it to be good community members. From a brief scan, it seems to me that a restructured implementation of rpc that lands in common should * not be tied up with eventlet on the consumer side * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) * not depend on a global singleton _RPC_IMPL * leave out/refactor some unused or non-general aspects, such as multicall, fanout_cast_to_server, notify, and fanout_cast (not so sure about that last one) It would also be useful if it had a way to subscribe to notification messages. From what I can tell, notifications don't work with any of the consumers in the existing drivers because those consumers all want to dispatch to a method invocation, but notifications aren't structured appropriately for that. I would like to abstract the messaging layer out of the existing nova RPC library and then build a compatible RPC layer on top of it. Notifications and other simple messages (such as meter data for ceilometer) would use the lower layer, and communication that truly works like RPC could use the higher level API. We should be able to build an agnostic RPC layer that uses the messaging driver, instead of having the two concepts bound up together. A very hearty +1. d ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
Hi Mark, On Thu, 2012-05-31 at 10:48 -0400, Mark Washenberger wrote: Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. This is a fairly general design question for openstack-common. If the behaviour of any given API should be dependent on the configuration of the service (i.e. in nova.conf, glance-api.conf, keystone.conf), then how should we design the API to handle that? Three options: 1) The API knows nothing about cfg: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 2) The API is passed a ConfigOpts instance: class Connection(object): def __init__(self, conf): self.cnx = self.connect(conf.broker_hostname, conf.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 3) The API uses the global config object class Connection(object): def __init__(self): self.cnx = self.connect(CONF.broker_hostname, CONF.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) I see (1) as having value if we want the API to be usable outside OpenStack. That might be worthwhile long term, but openstack-common's goal is to provide APIs shared between OpenStack projects. (2) and (3) have the huge benefit of configuration consistency - the same configuration keys, defaults, etc. shared across projects. (2) has value if there are cases where we want to use a non-global cfg object. Quantum might have this use case if it continues parsing plugin configuration separately from its main configuration. We'll see. (3) has value if all use cases are using a global object. I can see us getting to a point where we support both (2) and (3). I'm dubious of the value of (1) in the medium term, but maybe we might find a compelling use case for it. I'm also very happy to contribute modifications that would eliminate that connection. There are a few other reasons I'm concerned about nova's rpc implementation becoming the de facto standard. It has grown up organically and as such has some significant issues we should probably address before we create a precedent that other projects must adopt it to be good community members. other projects must adopt it to be good community members - that's an interesting take, not the way I'd put it. I think I'd say that if a project wants to do RPC, it makes more sense to improve the RPC API in openstack-common to meet its needs rather than start a from-scratch implementation or, worse, copy and paste what's there and customize it. From a brief scan, it seems to me that a restructured implementation of rpc that lands in common should * not be tied up with eventlet on the consumer side Sounds like a good and achievable goal. Is there or will there soon be an OpenStack project that needs has this requirement in order to be able to use this code? * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) What's the use case? * not depend on a global singleton _RPC_IMPL I agree. * leave out/refactor some unused or non-general aspects, such as multicall, fanout_cast_to_server, notify, and fanout_cast (not so sure about that last one) i.e. leave out stuff that is specific to Nova? How about if e.g. Cinder was using it? Don't forget that openstack-common has this notion of incubating APIs. Everything is in incubation at the moment. I'd say cfg is the closest to coming out of incubation. I expect to do a pretty detailed review of the patch to add rpc to openstack-common and make some API design recommendations. But I think we'll have to balance some of those design ideas between stuff we really should fix before it goes into openstack-common vs stuff that would need to be fixed
Re: [Openstack] cfg usage - option registration, global objects
On Thu, 2012-05-31 at 08:58 -0700, Johannes Erdfelt wrote: * let the consumer code decide when to ack a message (although maybe the concept of acking doesn't exist for all implementations?) My XenAPI idempotency branch delays ACKs until after it's done processing the message to ensure we get the message again after restart. https://review.openstack.org/#/c/7323 I can't say that I'm happy with the changes I've made to safely support delayed ACKs. Starting an instance is a long running operation. We confirm its completion by updating the instance state in the DB. We could switch to casting instance state change messages instead. Point is, I'd see an ack as message received, I've acted on it and handle the completion notification of long running messages separately. Or put it another way, an ack for a long running operation is similar to 202 Accepted in a REST API. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Thu, May 31, 2012 at 4:21 PM, Mark McLoughlin mar...@redhat.com wrote: Hi Mark, On Thu, 2012-05-31 at 10:48 -0400, Mark Washenberger wrote: Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I personally would consider a rigid connection between the rpc library and the configuration to be inappropriate in the context of openstack common. This is a fairly general design question for openstack-common. If the behaviour of any given API should be dependent on the configuration of the service (i.e. in nova.conf, glance-api.conf, keystone.conf), then how should we design the API to handle that? Three options: 1) The API knows nothing about cfg: class Connection(object): def __init__(self, broker_hostname, broker_port): self.cnx = self.connect(broker_hostname, broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 2) The API is passed a ConfigOpts instance: class Connection(object): def __init__(self, conf): self.cnx = self.connect(conf.broker_hostname, conf.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) 3) The API uses the global config object class Connection(object): def __init__(self): self.cnx = self.connect(CONF.broker_hostname, CONF.broker_port) def cast(self, topic, msg): self.cnx.cast(topic, msg) I see (1) as having value if we want the API to be usable outside OpenStack. That might be worthwhile long term, but openstack-common's goal is to provide APIs shared between OpenStack projects. (2) and (3) have the huge benefit of configuration consistency - the same configuration keys, defaults, etc. shared across projects. (2) has value if there are cases where we want to use a non-global cfg object. Quantum might have this use case if it continues parsing plugin configuration separately from its main configuration. We'll see. (3) has value if all use cases are using a global object. Also note that a configuration registry (per my previous email) can satisfy both approaches outlined in 2) and 3). d ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Thu, May 31, 2012, Mark McLoughlin mar...@redhat.com wrote: On Thu, 2012-05-31 at 08:58 -0700, Johannes Erdfelt wrote: My XenAPI idempotency branch delays ACKs until after it's done processing the message to ensure we get the message again after restart. https://review.openstack.org/#/c/7323 I can't say that I'm happy with the changes I've made to safely support delayed ACKs. Starting an instance is a long running operation. We confirm its completion by updating the instance state in the DB. We could switch to casting instance state change messages instead. Point is, I'd see an ack as message received, I've acted on it and handle the completion notification of long running messages separately. Or put it another way, an ack for a long running operation is similar to 202 Accepted in a REST API. My only concern with this is that it pushes durability of the message down a layer (assuming no orchestration). It's probably unavoidable to maintain flexibility in the RPC layer, but it does make things more complicated. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On 05/31/2012 04:21 PM, Mark McLoughlin wrote: I expect to do a pretty detailed review of the patch to add rpc to openstack-common and make some API design recommendations. But I think we'll have to balance some of those design ideas between stuff we really should fix before it goes into openstack-common vs stuff that would need to be fixed before the API comes out of incubation. I've been pushing the code with a goal that the submission to openstack-common is *only* a matter of changing the imports. Up to this point, it has mostly been a matter of removing dependencies on other parts of nova. We are actually at that point now ... unless there are other changes that are going to block it. I'd rather just work on improvements people would like to see while the code is in incubation in openstack-common. People are already copying it (in a feature branch for Quantum, Cinder, the Heat project, at least), so moving it as is will leave us in a better spot than we are now. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Hey, I had the chance to discuss the global conf issue with a good number of folks at the design summit and the conclusion I came away with was that opinions range from meh, it's fairly inelegant but I don't care much either way to I actually like the simplicity to we use global conf, it works and we're not changing. Indeed, at the common configuration patterns, there was very little in the way of opinion expressed at all: http://etherpad.openstack.org/FolsomCommonConfigPatterns Given that my goal here is to establish a common pattern used by all projects, that both Nova and Keystone uses global conf and there isn't a huge amount of interest in moving away from it, I'm proposing adding support for it to openstack-common: https://blueprints.launchpad.net/openstack-common/+spec/cfg-global-object Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Unfortunately, what this leads to is interdependencies between the openstack-common-rpc code and the openstack-common-cfg code. :( Now, if someone wants to use the openstack RPC code in their own project, they have to switch their way of configuring stuff to use global config objects. Tight coupling means less adherence to the do one thing and do it well mantra of *nix utilities and libraries and in general isn't good software design. I've posted patches to openstack-common, Nova, Glance and Keystone: https://review.openstack.org/#/q/status:open+branch:master+topic:bp/cfg-global-object,n,z Yes, I've noticed. :) I'm not a fan at all, but that said, it is more important to have consistency among the core projects IMHO, than to be ideologically pure on this point. Hopefully others who are being ideologically stubborn about things like the multiprocessing library is broken can see to also being less stubborn so certain patches can move forward in code review... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
Hey, I had the chance to discuss the global conf issue with a good number of folks at the design summit and the conclusion I came away with was that opinions range from meh, it's fairly inelegant but I don't care much either way to I actually like the simplicity to we use global conf, it works and we're not changing. Indeed, at the common configuration patterns, there was very little in the way of opinion expressed at all: http://etherpad.openstack.org/FolsomCommonConfigPatterns Given that my goal here is to establish a common pattern used by all projects, that both Nova and Keystone uses global conf and there isn't a huge amount of interest in moving away from it, I'm proposing adding support for it to openstack-common: https://blueprints.launchpad.net/openstack-common/+spec/cfg-global-object Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. I've posted patches to openstack-common, Nova, Glance and Keystone: https://review.openstack.org/#/q/status:open+branch:master+topic:bp/cfg-global-object,n,z Cheers, Mark. On Tue, 2012-03-06 at 10:10 +, Mark McLoughlin wrote: Hey, The original cfg design[1] assumed certain usage patterns that I hoped would be adopted by all projects using it. In gerrit, we're debating a set of patch to make keystone use these patterns: https://review.openstack.org/4547 I thought it was best to move some of that discussion here since I'm hoping we can get some rough consensus across projects. I really think it will be beneficial if we can share common idioms and patterns across projects, rather than just using the same library in different ways. [snip] Thread archived here: https://lists.launchpad.net/openstack/msg08329.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On Tue, May 29, 2012 at 4:04 AM, Mark McLoughlin mar...@redhat.com wrote: Hey, I had the chance to discuss the global conf issue with a good number of folks at the design summit and the conclusion I came away with was that opinions range from meh, it's fairly inelegant but I don't care much either way to I actually like the simplicity to we use global conf, it works and we're not changing. Indeed, at the common configuration patterns, there was very little in the way of opinion expressed at all: http://etherpad.openstack.org/FolsomCommonConfigPatterns Given that my goal here is to establish a common pattern used by all projects, that both Nova and Keystone uses global conf and there isn't a huge amount of interest in moving away from it, I'm proposing adding support for it to openstack-common: https://blueprints.launchpad.net/openstack-common/+spec/cfg-global-object Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If it can assume a global configuration object, things become much easier. Though not a fan of global objects, I've used (and still do use) them to simplify things. 99% of my usage of them in other projects are for configuration, as one might expect. I've avoided this in the past through passing parameters, but that often ends up quite messy and rather cumbersome to trace when debugging. More often than not, even when passed, configuration ends up getting used in contexts that are almost global anyway. If I do use a global object, I am deeply loathe to do it without the following: * ample documentation - how to use it, when not to use it, what one needs to do first, etc. * an API for it - no direct use of an object itself I have used zope.components to address the last point (the first one being addressed by good discipline ;-) ). It's simple to use: from zope.component import getGlobalSiteManager, getUtility from zope.interface import Interface class IConfig(Interface): A marker interface for configuration instances. gsm = getGlobalSiteManager() gsm.registerUtility(some_config_object, IConfig) That's done once, often in the __init__.py of a subpackage. All child modules and/or subpackages should need that configuration to be declared. Any code that doesn't need such configuration should be in a sibling (or higher) subpackage. Any time you need the config object: config = getUtility(IConfig) Note that you can register any object using this approach (including imported modules). The use of a marker interface may seem like overkill, but it provides a very natural place for documentation. Also, I usually wrap these calls in convenience functions in my projects, making use as simple, self-documenting, and maintainable as possible. More benefits to this approach (and why I chose to use it): I've got several projects (non-OpenStack) that have one or more other projects as their base -- the base project defines a configuration setup that is used in part or completely by the projects which depend upon it. zope.components let's me use a global registry to look up a config by interface, and this means that as long as a child project registers its config before the parent/base project code does, the base project code will be referencing the child projects configs. This does require strong import discipline, and careful subpackage organization, but I've found it a wonderful compromise for the global configuration problem. Note that the Pyramid project also uses the zope.components capabilities for configuration registry, so you might want to check out their code for potential use-cases. Hope this is helpful, d I've posted patches to openstack-common, Nova, Glance and Keystone: https://review.openstack.org/#/q/status:open+branch:master+topic:bp/cfg-global-object,n,z Cheers, Mark. On Tue, 2012-03-06 at 10:10 +, Mark McLoughlin wrote: Hey, The original cfg design[1] assumed certain usage patterns that I hoped would be adopted by all projects using it. In gerrit, we're debating a set of patch to make keystone use these patterns: https://review.openstack.org/4547 I thought it was best to move some of that discussion here since I'm hoping we can get some rough consensus across projects. I really think it will be beneficial if we can share common idioms and patterns across projects, rather than just using the same library in different ways. [snip] Thread archived here: https://lists.launchpad.net/openstack/msg08329.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to :
Re: [Openstack] cfg usage - option registration, global objects
Also, these global objects force us to do a bunch of hacks in unit tests. We need to do tricks to ensure the object is initialized as we want. We also need to save and restore its state between runs. I don't agree with the statement that they force a bunch of hacks, clearing state is a perfectly normal thing to do, it is done for any servers that get started, any mocks that are made, and every test database. Making sure that modifications to a configuration object are cleaned up is no different: there is no save and restore just always start from a blank slate and set things as required, same as in the non-global model. But ensuring that we are back at tabular rasa is difficult with global objects and running tests in a single process space. Many of the test cases are dependant on second order effects of function calls to configure internal objects, instead of being true unit tests that only call on a single object. This kind of refactoring is the norm in large projects, and leads to better tested code paths, reusuable objects, and objects that are easier to understand and track. One way that these types of things have been described is via the SOLID acronym. THis is a collection of best practices: http://en.wikipedia.org/wiki/SOLID_%28object-oriented_design%29 The Global config violates quite a few of these principals. By reading from a configuration object, you implicitly violate Single responsibility. Now it has at least two, one of which is figuring out how to construct and initialize itself. It also doesn't really follow the Open Closed principal: Once you get state from a Config object, you can no longer easily extend it, unless that extended object also gets its state from the same config. The big one is D: Dependency. By using a global config, you make dependencies on implementations, not abstractions. Dependency injection is pretty much impossible with a global config. Keystone really stands to benefit from reusability. An example: right now, we can only have one SQL Datasource, but it is likely that some users would have one data source for Identity and a different one for Tokens. That same set up has been describved for LDAP: Using a centralized LDAP server for Authentication, but a local one for Authorization. To do that, we have to split the LDAP config (and allow multiple) from the Identity config. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
Really not trying to derail, but Mark McLoughlin mar...@redhat.com said: [..] Also, these global objects force us to do a bunch of hacks in unit tests. We need to do tricks to ensure the object is initialized as we want. We also need to save and restore its state between runs. I don't agree with the statement that they force a bunch of hacks, clearing state is a perfectly normal thing to do, it is done for any servers that get started, any mocks that are made, and every test database. Making sure that modifications to a configuration object are cleaned up is no different: there is no save and restore just always start from a blank slate and set things as required, same as in the non-global model. Creating mocks, DB connection objects or re-starting servers on each test run is different from trying to restore a global object to its state before a test run. The CONF object is initialized the first time the config module is imported. Each test run you need to attempt to reset the object back to its original pristine state. Compared to just creating a config object for each run, I do think that's a hack. While I agree with what Mark is saying above, I really must point out that any significant shared state among unit tests is either a hack, or a smell, or most charitably, the wrong kind of test to run with unit tests. Any teardown that isn't just handled by the garbage collector is a good clue that we're doing something suboptimally. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
