Re: [openstack-dev] [Heat] Neutron resources and hidden dependencies
At the design summit we had a discussion[1] about redesigning Neutron resources to avoid the need for hidden dependencies (that is to say, dependencies which cannot be inferred from the template). Since then we've got close to fixing one of those issues[2] (thanks Bartosz!), but the patch is currently held up by a merge conflict because... we managed to add two more[3][4]. Sorry, I'm one of the reviewer of those, and [4] was a bit suspicious indeed, but it sounded like the best solution. [3] doesn't add dependencies, though. There's also another patch currently under review[5] that adds the most impressively complex hidden dependencies we've yet seen (though, fortunately, the worst of this is actually redundant and can be removed without effect). I know that due to the number of Neutron resources that currently override add_dependencies(), this may look like a normal part of a resource implementation. However, this is absolutely not the case. If you feel the need to override add_dependencies() for any reason then some part of the design is wrong. If you feel the need to do it for any reason other than not breaking existing soon-to-be-deprecated wrong designs (i.e. RouterGateway), then your part of the design is the part that's wrong. Core reviewers should treat any attempt to override add_dependencies() as a red flag IMO. It's unfortunate that many parts of the Neutron API are not great, especially that some pretty core functionality is currently balkanised into various 'extensions' that don't have a coherent interface. Nevertheless, that just means that we need to work harder to come up with resource designs that express the appropriate relationships between resources *in the template*, not with hidden relationships in the code. This means that orchestration will actually work regardless of whether all of the related resources are defined in the same template, and in fact regardless of whether they are defined in templates at all. I'd like to propose that we revert NetDHCPAgent and RouterL3Agent (which are somewhat misnamed, and serve only to connect a Net or Router to an existing agent, not to create an agent) and replace them with properties on the Net and Router classes, respectively. I'm not sure about NetDHCPAgent, as it doesn't add a dependency. Regarding RouterL3Agent, the only objection I have is if you want to use a router defined outside of the template, which if I remember correctly was the justification of this class. Do we have an answer for that? Thanks, -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] Neutron resources and hidden dependencies
On 18/12/13 05:21, Thomas Herve wrote: At the design summit we had a discussion[1] about redesigning Neutron resources to avoid the need for hidden dependencies (that is to say, dependencies which cannot be inferred from the template). Since then we've got close to fixing one of those issues[2] (thanks Bartosz!), but the patch is currently held up by a merge conflict because... we managed to add two more[3][4]. Sorry, I'm one of the reviewer of those, and [4] was a bit suspicious indeed, but it sounded like the best solution. [3] doesn't add dependencies, though. Quite right, my mistake, and I can definitely see how it would not have looked suspicious at the time. I can easily imagine though that there might be a bug raised in the future saying we need to add some magic dependencies to [3] to make sure the DHCP agent is selected before any instances/ports get created. Actually there's already an inline comment from the author in patch set 1 of [4] that says I will add dependency between subnet and NetDHCPAgent. There's also another patch currently under review[5] that adds the most impressively complex hidden dependencies we've yet seen (though, fortunately, the worst of this is actually redundant and can be removed without effect). I know that due to the number of Neutron resources that currently override add_dependencies(), this may look like a normal part of a resource implementation. However, this is absolutely not the case. If you feel the need to override add_dependencies() for any reason then some part of the design is wrong. If you feel the need to do it for any reason other than not breaking existing soon-to-be-deprecated wrong designs (i.e. RouterGateway), then your part of the design is the part that's wrong. Core reviewers should treat any attempt to override add_dependencies() as a red flag IMO. It's unfortunate that many parts of the Neutron API are not great, especially that some pretty core functionality is currently balkanised into various 'extensions' that don't have a coherent interface. Nevertheless, that just means that we need to work harder to come up with resource designs that express the appropriate relationships between resources *in the template*, not with hidden relationships in the code. This means that orchestration will actually work regardless of whether all of the related resources are defined in the same template, and in fact regardless of whether they are defined in templates at all. I'd like to propose that we revert NetDHCPAgent and RouterL3Agent (which are somewhat misnamed, and serve only to connect a Net or Router to an existing agent, not to create an agent) and replace them with properties on the Net and Router classes, respectively. I'm not sure about NetDHCPAgent, as it doesn't add a dependency. That's true, although I suspect it may only be working by coincidence now. It also seems to me that we should have a consistent approach for selecting agents. This resource does nothing but connect two other resources together in a way that, if it makes any difference at all, makes the dependency resolution worse not better. (Contrast this with VolumeAttachment or EIPAssociation, which exist to solve some specific dependency issues.) Conceptually, it is just a property of the Network IMO. Regarding RouterL3Agent, the only objection I have is if you want to use a router defined outside of the template, which if I remember correctly was the justification of this class. Do we have an answer for that? I haven't seen that justification advanced anywhere in relation to this particular resource type, but maybe I missed it. In any event, if you want to use a Router defined outside of the template, you should also assign it to the correct agent outside of the template. It's difficult to get upset about that, especially given that it needs to be done before you use the Router for anything anyway. Basically there's no there's no safe way to do this other than at the time you create the router... that's the nice thing about not hacking the dependencies - when they're all explicit in the template then everything works the same way whether it's created in Heat or not, in the same template or not. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] Neutron resources and hidden dependencies
At the design summit we had a discussion[1] about redesigning Neutron resources to avoid the need for hidden dependencies (that is to say, dependencies which cannot be inferred from the template). Since then we've got close to fixing one of those issues[2] (thanks Bartosz!), but the patch is currently held up by a merge conflict because... we managed to add two more[3][4]. There's also another patch currently under review[5] that adds the most impressively complex hidden dependencies we've yet seen (though, fortunately, the worst of this is actually redundant and can be removed without effect). I know that due to the number of Neutron resources that currently override add_dependencies(), this may look like a normal part of a resource implementation. However, this is absolutely not the case. If you feel the need to override add_dependencies() for any reason then some part of the design is wrong. If you feel the need to do it for any reason other than not breaking existing soon-to-be-deprecated wrong designs (i.e. RouterGateway), then your part of the design is the part that's wrong. Core reviewers should treat any attempt to override add_dependencies() as a red flag IMO. It's unfortunate that many parts of the Neutron API are not great, especially that some pretty core functionality is currently balkanised into various 'extensions' that don't have a coherent interface. Nevertheless, that just means that we need to work harder to come up with resource designs that express the appropriate relationships between resources *in the template*, not with hidden relationships in the code. This means that orchestration will actually work regardless of whether all of the related resources are defined in the same template, and in fact regardless of whether they are defined in templates at all. I'd like to propose that we revert NetDHCPAgent and RouterL3Agent (which are somewhat misnamed, and serve only to connect a Net or Router to an existing agent, not to create an agent) and replace them with properties on the Net and Router classes, respectively. The Route resource has not yet been merged, so we can discuss in the review the best course of action - which IMO is to: (a) Fix the Neutron API to allow atomic updates to the route table; and (b) Use a reference to a RouterInterface as the nexthop value, to ensure that routes are not created before their nexthops are available. cheers, Zane. [1] https://etherpad.openstack.org/p/icehouse-summit-heat-exorcism [2] https://review.openstack.org/#/c/60118/ [3] https://review.openstack.org/#/c/59626/8 [4] https://review.openstack.org/#/c/61388/3 [5] https://review.openstack.org/#/c/41044/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
