Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Zane Bitter]

On 01/04/16 12:31, Steven Hardy wrote:

On Fri, Apr 01, 2016 at 04:15:30PM +0200, Thomas Herve wrote:

On Fri, Apr 1, 2016 at 3:21 AM, Zane Bitter  wrote:

On 31/03/16 18:10, Zane Bitter wrote:



I'm in favour of some sort of variable-based implementation for a few
reasons. One is that (5) seems to come up fairly regularly in a complex
deployment like TripleO. Another is that Fn::If feels awkward compared
to get_variable.



I actually have to revise this last part after reviewing the patches.
get_variable can't replace Fn::If, because we'd still need to handle stuff
of the form:

 some_property: {if: [{get_variable: some_var},
  {get_resource: res1},
  {get_resource: res2}]

where the alternatives can't come from a variable because they contain
resource references and we have said we'd constrain variables to be static.

In fact the intrinsic functions that could be allowed in the first argument
to the {if: } function would have to constrained in the same way as the
constraint field in the resource, because we should only validate and obtain
dependencies from _one_ of the alternates, so we need to be able to
determine which one statically and not have to wait until the actual value
is resolved. This is possibly the strongest argument for keeping on the cfn
implementation course.


We talked about another possibilities on IRC: instead of having a new
section, create a new resource OS::Heat::Value which can hold some
data. It would look like that:

resources:
 is_prod:
 type: OS::Heat::Value
 properties:
 value: {equals: {get_param, env}, prod}}

 my_resource:
 condition: {get_attr: [is_prod, value}}


Another alternative (which maybe you discussed, sorry I missed the IRC
chat) would be just to use parameters, as that's already conceptually where
we obtain values that are input to resources.

E.g:

parameters:
   env:
type: string
default: prod

   is_prod:
 type: boolean
 default: {equals: {get_param, env}}

 From an interface standpoint this seems much cleaner and more intuitive than
the other solutions discussed IMHO, but I suspect it's potentially harder to
implement.


Yeah, the problems to solve here would be:

1) Need to define some subset of functions that are valid in templates 
(for HOT, probably all except get_resource and get_attr).


  - Easy enough, just add a separate method called param_functions or 
something to the Template class. We'd have to do the same for the 
"conditions" section anyway.


2) Need to prevent circular references between parameters.

  - Hard. Since currently no templates support parsing parameters for 
functions, we can add a separate implementation without breaking the 
third-party Template API though, so that helps. Lots of edge cases though.


3) Need to make clear to users that only a parameter is a valid input to 
a resource condition.


  - Could be done with something like:

 my_resource:
   condition_parameter: is_prod

instead of:

 my_resource:
   condition: {get_param: is_prod}

and something similar for {if: [is_prod, , ]}

4) Need to implement the conditions section in CloudFormation templates.

  - Could have some sort of hacky pseudo-parameter thing.


Your original example gets much cleaner too if we allow all intrinsic function
(except get_attr) in the scope of parameters:

parameters:
   host:
 type: string
   port:
 type: string
   endpoint:
 type: string
 default:
   str_replace:
 template:
http://HOSTORT/
 params:
HOST: {get_param: host}
PORT: {get_param: port}


So the alternative is:

  parameters:
host:
  type: string
port:
  type: string

  resources:
endpoint:
  type: OS::Heat::Value
  properties:
type: string
value:
  str_replace:
template:
  http://HOST:PORT/
params:
  HOST: {get_param: host}
  PORT: {get_param: port}

which is known to be trivial to implement, and has the advantage that it 
can also combine in data that is only available at runtime (e.g. it's 
easy to imagine that HOST might come from within the template, but that 
you'd still need to use the endpoint URL in enough places that it's not 
worth copying and pasting the whole str_replace function). Also, every 
template format would get it for free, which is nice.


The only downside is that you can't replace the whole value definition 
with a parameter value passed in from outside. Given that it would have 
to be a complete override including the substitute values (i.e. we can't 
parse functions from template _values_, only defaults), I'm not sure 
that is a big loss.


In my view it is almost a toss-up, but I am marginally in favour of a 
conditions section (per the current proposed implementation) plus a 
OS::Heat::Value resource over computable parameter 

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Zane Bitter]

On 05/04/16 06:43, Steven Hardy wrote:

On Fri, Apr 01, 2016 at 04:39:02PM +, Fox, Kevin M wrote:

Why is imperative programming always brought up when discussing
conditionals in the templates? We are not wanting anything imperative. The
heat engine still picks the final ordering of things. We just want to give
it all the valid options, and then enough knowledge to eliminate
unacceptlable solutions. Maybe we are using the wrong term for them? Would
'constraint' suit the conversation better?


Since nobody is really disputing that we should implement *something* 
can we maybe just stop having this unhelpful argument about semantics?



I think it's because as soon as you start conflating specific conditions
with a declarative model (in the same place, e.g within the same template),
you end up with an imperative flow whether you want it or not.


Sorry, but all of the templates and environment files and other data 
(including parameters) together constitute the model. The distinction 
you're trying to draw between stuff that is defined in the template and 
stuff that is defined in the environment (as in your "more declarative 
approach" below) is entirely spurious.



E.g if condition=$foo, resource X is created, but resource Y isn't, which
means you no longer have an explicit definition of the required state in
the template.


You could make a similar argument about parameters, and it would be just 
as pointless.



This feels imperative because the template now contains
statements which change the resulting desired state and it influences
references (e.g attributes) elsewhere in the template.


As long as we follow the cfn model, or something in the same ballpark, 
it's completely declarative. You're *declaring* that you want this set 
of things, and simultaneously *declaring* that you don't want this other 
set of things. The only difference between this and the current 
templates is that you're being oddly specific about things you don't want :D


There's simply no hint of imperativeness here - the conditions are all 
completely determined before the model is built.


The same would arguably not be true if we allowed arbitrary data to be 
passed as the condition and for it to be evaluated at runtime (e.g. from 
get_attr), and the need to have some distinction in the template 
structure to indicate to users that we cannot, in fact, allow this is 
the reason that I now support hewing close to the cfn model.



The more declarative approach to composition I was aiming for here:

https://github.com/openstack/heat-specs/blob/master/specs/mitaka/resource-capabilities.rst

This is a model where each template retains it's declarative interface, and
you select which combination of templates to use by "eliminating
unacceptlable solutions" exactly as you say above.


Two problems with having this as the _only_ solution:

1) Stacks are an extremely heavyweight abstraction. This is part of the 
reason that Heat can chew up 10s of GB of RAM on a TripleO deployment 
with several hundred nodes.
2) Because moving a resource into a nested stack severs its dependency 
relationship with the surrounding resources, the declarative model is 
actually broken when it comes to deleting or replacing the now-nested 
resource.[1]


cheers,
Zane.

[1] https://etherpad.openstack.org/p/mitaka-heat-break-stack-barrier

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Steven Hardy]

On Fri, Apr 01, 2016 at 04:39:02PM +, Fox, Kevin M wrote:
>Why is imperative programming always brought up when discussing
>conditionals in the templates? We are not wanting anything imperative. The
>heat engine still picks the final ordering of things. We just want to give
>it all the valid options, and then enough knowledge to eliminate
>unacceptlable solutions. Maybe we are using the wrong term for them? Would
>'constraint' suit the conversation better?

I think it's because as soon as you start conflating specific conditions
with a declarative model (in the same place, e.g within the same template),
you end up with an imperative flow whether you want it or not.

E.g if condition=$foo, resource X is created, but resource Y isn't, which
means you no longer have an explicit definition of the required state in
the template. This feels imperative because the template now contains
statements which change the resulting desired state and it influences
references (e.g attributes) elsewhere in the template.

The more declarative approach to composition I was aiming for here:

https://github.com/openstack/heat-specs/blob/master/specs/mitaka/resource-capabilities.rst

This is a model where each template retains it's declarative interface, and
you select which combination of templates to use by "eliminating
unacceptlable solutions" exactly as you say above.

Sadly I've not made much progress on this as some folks opposed it and
despite a broadly positive summit session it wasn't approved until late in
the Mitaka cycle :( Maybe it's something we can revisit in Newton tho.

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Qiming Teng]

 
> parameters:
>   env:
>type: string
>default: prod
> 
>   is_prod:
> type: boolean
> default: {equals: {get_param, env}}
> 
> From an interface standpoint this seems much cleaner and more intuitive than
> the other solutions discussed IMHO, but I suspect it's potentially harder to
> implement.
> 
> Your original example gets much cleaner too if we allow all intrinsic function
> (except get_attr) in the scope of parameters:
> 
> parameters:
>   host:
> type: string
>   port:
> type: string
>   endpoint:
> type: string
> default:
>   str_replace:
> template:
>http://HOSTORT/
> params:
>HOST: {get_param: host}
>PORT: {get_param: port}
> 
> Steve

+1 to restricting the variable evaluation into the 'parameters' section
which we can treat as the 'static' part of a template. Maybe that is one
of the reasons why aws has a 'conditions' section to avoid getting into
the lazy evaluation mess?

Regards,
  Qiming


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Fox, Kevin M]

Params have the added advantage that horizon could be potentially tweaked to do 
form validation in the ui based in it.

Thanks,
Kevin


From: Steven Hardy
Sent: Friday, April 01, 2016 9:31:45 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

On Fri, Apr 01, 2016 at 04:15:30PM +0200, Thomas Herve wrote:
> On Fri, Apr 1, 2016 at 3:21 AM, Zane Bitter <zbit...@redhat.com> wrote:
> > On 31/03/16 18:10, Zane Bitter wrote:
> >>
> >>
> >> I'm in favour of some sort of variable-based implementation for a few
> >> reasons. One is that (5) seems to come up fairly regularly in a complex
> >> deployment like TripleO. Another is that Fn::If feels awkward compared
> >> to get_variable.
> >
> >
> > I actually have to revise this last part after reviewing the patches.
> > get_variable can't replace Fn::If, because we'd still need to handle stuff
> > of the form:
> >
> > some_property: {if: [{get_variable: some_var},
> >  {get_resource: res1},
> >  {get_resource: res2}]
> >
> > where the alternatives can't come from a variable because they contain
> > resource references and we have said we'd constrain variables to be static.
> >
> > In fact the intrinsic functions that could be allowed in the first argument
> > to the {if: } function would have to constrained in the same way as the
> > constraint field in the resource, because we should only validate and obtain
> > dependencies from _one_ of the alternates, so we need to be able to
> > determine which one statically and not have to wait until the actual value
> > is resolved. This is possibly the strongest argument for keeping on the cfn
> > implementation course.
>
> We talked about another possibilities on IRC: instead of having a new
> section, create a new resource OS::Heat::Value which can hold some
> data. It would look like that:
>
> resources:
> is_prod:
> type: OS::Heat::Value
> properties:
> value: {equals: {get_param, env}, prod}}
>
> my_resource:
> condition: {get_attr: [is_prod, value}}

Another alternative (which maybe you discussed, sorry I missed the IRC
chat) would be just to use parameters, as that's already conceptually where
we obtain values that are input to resources.

E.g:

parameters:
  env:
   type: string
   default: prod

  is_prod:
type: boolean
default: {equals: {get_param, env}}

From an interface standpoint this seems much cleaner and more intuitive than
the other solutions discussed IMHO, but I suspect it's potentially harder to
implement.

Your original example gets much cleaner too if we allow all intrinsic function
(except get_attr) in the scope of parameters:

parameters:
  host:
type: string
  port:
type: string
  endpoint:
type: string
default:
  str_replace:
template:
   http://HOSTORT/
params:
   HOST: {get_param: host}
   PORT: {get_param: port}

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Fox, Kevin M]

Why is imperative programming always brought up when discussing conditionals in 
the templates? We are not wanting anything imperative. The heat engine still 
picks the final ordering of things. We just want to give it all the valid 
options, and then enough knowledge to eliminate unacceptlable solutions. Maybe 
we are using the wrong term for them? Would 'constraint' suit the conversation 
better?

Thanks,
Kevin


From: Thomas Herve
Sent: Thursday, March 31, 2016 7:10:45 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

On Thu, Mar 31, 2016 at 2:25 PM, Huangtianhua <huangtian...@huawei.com> wrote:
> The conditions function has been requested for a long time, and there have 
> been several previous discussions, which all ended up in debating the 
> implementation, and no result.
> https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
> https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst

And for a reason: this is a tricky issue, and introducing imperative
constructs in a template can lead to bad practices.

> I think we should focus on the simplest possible way(same as AWS) to meet the 
> user requirement, and follows the AWS, there is no doubt that we will get a 
> very good compatibility.
> And the patches are good in-progress. I don't want everything back to zero:)
> https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function

I don't say that you should scratch everything. I'm mostly OK with
what has been done, with the exception of the top-level conditions
section. Templates are our user interface, and we need to be very
careful when we introduce new things. 3 years ago following AWS was
the easy path because we didn't have much idea on what to do, but I
believe we now have enough background to be more innovative.

It's also slightly worrying that the spec "only" got 3 cores approving
it, especially on such a touchy subject. I'm guilty as others to not
have voiced my concerns then, though.

--
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Steven Hardy]

On Fri, Apr 01, 2016 at 04:15:30PM +0200, Thomas Herve wrote:
> On Fri, Apr 1, 2016 at 3:21 AM, Zane Bitter  wrote:
> > On 31/03/16 18:10, Zane Bitter wrote:
> >>
> >>
> >> I'm in favour of some sort of variable-based implementation for a few
> >> reasons. One is that (5) seems to come up fairly regularly in a complex
> >> deployment like TripleO. Another is that Fn::If feels awkward compared
> >> to get_variable.
> >
> >
> > I actually have to revise this last part after reviewing the patches.
> > get_variable can't replace Fn::If, because we'd still need to handle stuff
> > of the form:
> >
> > some_property: {if: [{get_variable: some_var},
> >  {get_resource: res1},
> >  {get_resource: res2}]
> >
> > where the alternatives can't come from a variable because they contain
> > resource references and we have said we'd constrain variables to be static.
> >
> > In fact the intrinsic functions that could be allowed in the first argument
> > to the {if: } function would have to constrained in the same way as the
> > constraint field in the resource, because we should only validate and obtain
> > dependencies from _one_ of the alternates, so we need to be able to
> > determine which one statically and not have to wait until the actual value
> > is resolved. This is possibly the strongest argument for keeping on the cfn
> > implementation course.
> 
> We talked about another possibilities on IRC: instead of having a new
> section, create a new resource OS::Heat::Value which can hold some
> data. It would look like that:
> 
> resources:
> is_prod:
> type: OS::Heat::Value
> properties:
> value: {equals: {get_param, env}, prod}}
> 
> my_resource:
> condition: {get_attr: [is_prod, value}}

Another alternative (which maybe you discussed, sorry I missed the IRC
chat) would be just to use parameters, as that's already conceptually where
we obtain values that are input to resources.

E.g:

parameters:
  env:
   type: string
   default: prod

  is_prod:
type: boolean
default: {equals: {get_param, env}}

>From an interface standpoint this seems much cleaner and more intuitive than
the other solutions discussed IMHO, but I suspect it's potentially harder to
implement.

Your original example gets much cleaner too if we allow all intrinsic function
(except get_attr) in the scope of parameters:

parameters:
  host:
type: string
  port:
type: string
  endpoint:
type: string
default:
  str_replace:
template:
   http://HOSTORT/
params:
   HOST: {get_param: host}
   PORT: {get_param: port}

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Zane Bitter]

On 01/04/16 10:15, Thomas Herve wrote:

On Fri, Apr 1, 2016 at 3:21 AM, Zane Bitter  wrote:

On 31/03/16 18:10, Zane Bitter wrote:



I'm in favour of some sort of variable-based implementation for a few
reasons. One is that (5) seems to come up fairly regularly in a complex
deployment like TripleO. Another is that Fn::If feels awkward compared
to get_variable.



I actually have to revise this last part after reviewing the patches.
get_variable can't replace Fn::If, because we'd still need to handle stuff
of the form:

 some_property: {if: [{get_variable: some_var},
  {get_resource: res1},
  {get_resource: res2}]

where the alternatives can't come from a variable because they contain
resource references and we have said we'd constrain variables to be static.

In fact the intrinsic functions that could be allowed in the first argument
to the {if: } function would have to constrained in the same way as the
constraint field in the resource, because we should only validate and obtain
dependencies from _one_ of the alternates, so we need to be able to
determine which one statically and not have to wait until the actual value
is resolved. This is possibly the strongest argument for keeping on the cfn
implementation course.


We talked about another possibilities on IRC: instead of having a new
section, create a new resource OS::Heat::Value which can hold some
data. It would look like that:

resources:
 is_prod:
 type: OS::Heat::Value
 properties:
 value: {equals: {get_param, env}, prod}}


We could even get fancy and have an optional 'type' property for type 
checking.



 my_resource:
 condition: {get_attr: [is_prod, value}}


Ah, I think I misunderstood the conversation on IRC.

Using a value from get_attr for a condition like this doesn't work, at 
least with something similar to the current implementation, because the 
result would only be known once the is_prod resource is created, but 
currently at least we exclude my_resource from the list of resource 
definitions that we return when we ask the template what it contains, 
long before we create any resources. It's possible to imagine a world in 
which all of the resource definitions are returned and we decide at the 
point of creating/updating what to do with them (create/not 
create/delete), but it would require extensive changes throughout Heat. 
The current proposed implementation is much, much safer.


What this would be *great* for is to provide the variable functionality 
that things like TripleO could really use, without needing to affect the 
implementation of conditionals at all.



Sounds like it would be fairly flexible, really easy to implement, and
solve most of the issues I can think of (including being included in
the dependency resolution mechanism for free).


Adding OS::Heat::Value for variables & sticking with the existing spec 
for the conditional resources gets my +1. Seems less complicated all round.


cheers,
Zane.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Thomas Herve]

On Fri, Apr 1, 2016 at 3:21 AM, Zane Bitter  wrote:
> On 31/03/16 18:10, Zane Bitter wrote:
>>
>>
>> I'm in favour of some sort of variable-based implementation for a few
>> reasons. One is that (5) seems to come up fairly regularly in a complex
>> deployment like TripleO. Another is that Fn::If feels awkward compared
>> to get_variable.
>
>
> I actually have to revise this last part after reviewing the patches.
> get_variable can't replace Fn::If, because we'd still need to handle stuff
> of the form:
>
> some_property: {if: [{get_variable: some_var},
>  {get_resource: res1},
>  {get_resource: res2}]
>
> where the alternatives can't come from a variable because they contain
> resource references and we have said we'd constrain variables to be static.
>
> In fact the intrinsic functions that could be allowed in the first argument
> to the {if: } function would have to constrained in the same way as the
> constraint field in the resource, because we should only validate and obtain
> dependencies from _one_ of the alternates, so we need to be able to
> determine which one statically and not have to wait until the actual value
> is resolved. This is possibly the strongest argument for keeping on the cfn
> implementation course.

We talked about another possibilities on IRC: instead of having a new
section, create a new resource OS::Heat::Value which can hold some
data. It would look like that:

resources:
is_prod:
type: OS::Heat::Value
properties:
value: {equals: {get_param, env}, prod}}

my_resource:
condition: {get_attr: [is_prod, value}}


Sounds like it would be fairly flexible, really easy to implement, and
solve most of the issues I can think of (including being included in
the dependency resolution mechanism for free).

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Zane Bitter]

On 31/03/16 18:10, Zane Bitter wrote:


I'm in favour of some sort of variable-based implementation for a few
reasons. One is that (5) seems to come up fairly regularly in a complex
deployment like TripleO. Another is that Fn::If feels awkward compared
to get_variable.


I actually have to revise this last part after reviewing the patches. 
get_variable can't replace Fn::If, because we'd still need to handle 
stuff of the form:


some_property: {if: [{get_variable: some_var},
 {get_resource: res1},
 {get_resource: res2}]

where the alternatives can't come from a variable because they contain 
resource references and we have said we'd constrain variables to be static.


In fact the intrinsic functions that could be allowed in the first 
argument to the {if: } function would have to constrained in the same 
way as the constraint field in the resource, because we should only 
validate and obtain dependencies from _one_ of the alternates, so we 
need to be able to determine which one statically and not have to wait 
until the actual value is resolved. This is possibly the strongest 
argument for keeping on the cfn implementation course.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Zane Bitter]

On 31/03/16 10:10, Thomas Herve wrote:

On Thu, Mar 31, 2016 at 2:25 PM, Huangtianhua  wrote:

The conditions function has been requested for a long time, and there have been 
several previous discussions, which all ended up in debating the 
implementation, and no result.
https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst


I wouldn't say no result, all of those discussions ended up moving in 
the direction of the variables thing that Thomas is suggesting here.



And for a reason: this is a tricky issue, and introducing imperative
constructs in a template can lead to bad practices.


Yes, the disagreement was largely over whether we should do it at all.


I think we should focus on the simplest possible way(same as AWS) to meet the 
user requirement, and follows the AWS, there is no doubt that we will get a 
very good compatibility.
And the patches are good in-progress. I don't want everything back to zero:)
https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function


I'm sympathetic to this. Happily, I don't think the change being 
proposed is even close to the level of 'back to zero' :)



I don't say that you should scratch everything. I'm mostly OK with
what has been done, with the exception of the top-level conditions
section. Templates are our user interface, and we need to be very
careful when we introduce new things. 3 years ago following AWS was
the easy path because we didn't have much idea on what to do, but I
believe we now have enough background to be more innovative.


+1

I think looking at the cfn implementation has an unfortunate effect of 
making us look at a number of separate problems as if they were a 
single, monolithic one. The problems it solves are:


1) How to conditionally disable a resource
2) How to conditionally change data (e.g. properties) without requiring 
the user to input all of the data in a parameter
3) How to share the same condition in multiple resource without having 
to duplicate the condition logic
4) How to share the same condition for multiple pieces of data (or share 
between data and resources) without having to duplicate the condition logic


The solution cfn has is a "Conditions" section that contains the logic 
(3) & (4), a "Condition" on a resource that refers to an entry in the 
"Conditions" section (1) and a Fn::If intrinsic function that also 
refers to an entry in the "Conditions" section (2).


An obvious problem with (3)/(4) is that they eliminate the need for 
duplicating the condition logic in more than one place, but if you need 
the data that is selected by Fn::If in more than one place then you'll 
end up repeating _that_. Let's call this (5).


A solution using variables (I prefer "values" but I'm happy to bikeshed) 
instead of conditions would solve (2) and (5). (1) and (3) can obviously 
be solved by a "condition" in a resource that takes a value directly 
(instead of just a condition name).


Importantly, (4) can be achieved only if the get_variable function is 
allowed inside variable definitions, and that means we need to detect 
circular references. (That'd be a fun little programming problem for 
someone, but not a show-stopper.) You could argue that if you have (3) 
and (5) then (4) is not as important, since the condition in many cases 
might be just a parameter reference without any more complicated logic.



The reason (I speculate) that cfn went with the implementation they did, 
specifically having resource conditions contain the *name* of a 
condition variable rather than a value, is that they probably didn't 
want to have to parse the different parts of the resource definition 
differently. Specifically, the conditions must be resolved statically at 
therefore stuff like Fn::GetAtt are not valid. It's easy to explain a 
different section of the template where those are not valid, but much 
harder to explain that they're valid in some parts of the resource 
definition but not others.


We could get around this by calling the field condition_variable or 
something and having it contain a name of a variable rather than an 
explicit get_variable function, so that any conditions you want to write 
have to go into a variable, but not all variables would have to be 
conditions.



I'm in favour of some sort of variable-based implementation for a few 
reasons. One is that (5) seems to come up fairly regularly in a complex 
deployment like TripleO. Another is that Fn::If feels awkward compared 
to get_variable.


But maybe the biggest one is that what we choose here is not only our 
public facing user-interface, as Thomas pointed out, but also the API 
interface against which people (including our future selves) can write 
their own template format plugins. The AWS solution can be trivially 
implemented in terms of the variable-based 

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Fox, Kevin M]

+1. This sounds good. The lack of any conditionals at all has caused a lot of 
pain.

Thanks,
Kevin


From: Huangtianhua
Sent: Thursday, March 31, 2016 5:25:29 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

The conditions function has been requested for a long time, and there have been 
several previous discussions, which all ended up in debating the 
implementation, and no result.
https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst

I think we should focus on the simplest possible way(same as AWS) to meet the 
user requirement, and follows the AWS, there is no doubt that we will get a 
very good compatibility.
And the patches are good in-progress. I don't want everything back to zero:)
https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function

In the example you given of 'variables', seems there's no relation with 
resource/output/property conditions, it seems as another function which likes 
really 'variables' to used in template.

-邮件原件-
发件人: Thomas Herve [mailto:the...@redhat.com]
发送时间: 2016年3月31日 19:55
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [Heat] Re-evaluate conditions specification

On Thu, Mar 31, 2016 at 10:40 AM, Thomas Herve <the...@redhat.com> wrote:
> Hi all,
>
> As the patches for conditions support are incoming, I've found
> something in the code (and the spec) I'm not really happy with. We're
> creating a new top-level section in the template called "conditions"
> which holds names that can be reused for conditionally creating
> resource.
>
> While it's fine and maps to what AWS does, I think it's a bit
> short-sighted and limited. What I have suggested in the past is to
> have a "variables" (or whatever you want to call it) section, where
> one can declare names and values. Then we can add an intrinsic
> function to retrieve data from there, and use that for examples for
> conditions.

I was asked to give examples, here's at least one that can illustrate what I 
meant:

parameters:
   host:
  type: string
   port:
  type: string

variables:
   endpoint:
  str_replace:
template:
   http://HOST:PORT/
params:
   HOST: {get_param: host}
   PORT: {get_param: port}

resources:
   config1:
  type: OS::Heat::StructuredConfig
  properties:
config:
   hosts: [{get_variable: endpoint}]

--
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Qiming Teng]

On Thu, Mar 31, 2016 at 09:21:43AM -0400, Rabi Mishra wrote:
 
> If I understand the suggestion correctly, the only relation it has with 
> conditions is,
> conditions are nothing but variables(boolean).
> 
> conditions: {
> 'for_prod': {equals: [{get_param: env_type}, 'prod']}
>   }
> 
> would be
> 
> variables:
>for_prod: {equals: [{get_param: env_type}, prod]}
> 
> 
> then you can use it in your example as:
> 
> floating_ip:
>   type: OS::Nova::FloatingIP
>   condition: {get_variable: for_prod}
> 
> so suggestion is to make it more generic, so that it can be used for other 
> things
> and reduce some of the verbosity in the templates.
> 
> However, I think the term 'variable' makes is sound more like a programming 
> thing. May
> be we can use something better. However, personally I kind of like the idea.

well ... now I get a better idea about the suggestion. Actually,
'variables' is not that bad in my opinion.

Regards,
 Qiming


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Thomas Herve]

On Thu, Mar 31, 2016 at 2:25 PM, Huangtianhua  wrote:
> The conditions function has been requested for a long time, and there have 
> been several previous discussions, which all ended up in debating the 
> implementation, and no result.
> https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
> https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst

And for a reason: this is a tricky issue, and introducing imperative
constructs in a template can lead to bad practices.

> I think we should focus on the simplest possible way(same as AWS) to meet the 
> user requirement, and follows the AWS, there is no doubt that we will get a 
> very good compatibility.
> And the patches are good in-progress. I don't want everything back to zero:)
> https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function

I don't say that you should scratch everything. I'm mostly OK with
what has been done, with the exception of the top-level conditions
section. Templates are our user interface, and we need to be very
careful when we introduce new things. 3 years ago following AWS was
the easy path because we didn't have much idea on what to do, but I
believe we now have enough background to be more innovative.

It's also slightly worrying that the spec "only" got 3 cores approving
it, especially on such a touchy subject. I'm guilty as others to not
have voiced my concerns then, though.

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Rabi Mishra]

> The conditions function has been requested for a long time, and there have
> been several previous discussions, which all ended up in debating the
> implementation, and no result.
> https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
> https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst
> 
> I think we should focus on the simplest possible way(same as AWS) to meet the
> user requirement, and follows the AWS, there is no doubt that we will get a
> very good compatibility.
> And the patches are good in-progress. I don't want everything back to zero:)
> https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function
> 
> In the example you given of 'variables', seems there's no relation with
> resource/output/property conditions, it seems as another function which
> likes really 'variables' to used in template.

If I understand the suggestion correctly, the only relation it has with 
conditions is,
conditions are nothing but variables(boolean).

conditions: {
'for_prod': {equals: [{get_param: env_type}, 'prod']}
  }

would be

variables:
   for_prod: {equals: [{get_param: env_type}, prod]}


then you can use it in your example as:

floating_ip:
  type: OS::Nova::FloatingIP
  condition: {get_variable: for_prod}

so suggestion is to make it more generic, so that it can be used for other 
things
and reduce some of the verbosity in the templates.

However, I think the term 'variable' makes is sound more like a programming 
thing. May
be we can use something better. However, personally I kind of like the idea.
 
> -邮件原件-
> 发件人: Thomas Herve [mailto:the...@redhat.com]
> 发送时间: 2016年3月31日 19:55
> 收件人: OpenStack Development Mailing List (not for usage questions)
> 主题: Re: [openstack-dev] [Heat] Re-evaluate conditions specification
> 
> On Thu, Mar 31, 2016 at 10:40 AM, Thomas Herve  wrote:
> > Hi all,
> >
> > As the patches for conditions support are incoming, I've found
> > something in the code (and the spec) I'm not really happy with. We're
> > creating a new top-level section in the template called "conditions"
> > which holds names that can be reused for conditionally creating
> > resource.
> >
> > While it's fine and maps to what AWS does, I think it's a bit
> > short-sighted and limited. What I have suggested in the past is to
> > have a "variables" (or whatever you want to call it) section, where
> > one can declare names and values. Then we can add an intrinsic
> > function to retrieve data from there, and use that for examples for
> > conditions.
> 
> I was asked to give examples, here's at least one that can illustrate what I
> meant:
> 
> parameters:
>host:
>   type: string
>port:
>   type: string
> 
> variables:
>endpoint:
>   str_replace:
> template:
>http://HOST:PORT/
> params:
>HOST: {get_param: host}
>PORT: {get_param: port}
> 
> resources:
>config1:
>   type: OS::Heat::StructuredConfig
>   properties:
> config:
>hosts: [{get_variable: endpoint}]
> 
> --
> Thomas
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] 答复: [Heat] Re-evaluate conditions specification

[Huangtianhua]

The conditions function has been requested for a long time, and there have been 
several previous discussions, which all ended up in debating the 
implementation, and no result.
https://review.openstack.org/#/c/84468/3/doc/source/template_guide/hot_spec.rst
https://review.openstack.org/#/c/153771/1/specs/kilo/resource-enabled-meta-property.rst

I think we should focus on the simplest possible way(same as AWS) to meet the 
user requirement, and follows the AWS, there is no doubt that we will get a 
very good compatibility.
And the patches are good in-progress. I don't want everything back to zero:)
https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bp/support-conditions-function

In the example you given of 'variables', seems there's no relation with 
resource/output/property conditions, it seems as another function which likes 
really 'variables' to used in template.

-邮件原件-
发件人: Thomas Herve [mailto:the...@redhat.com] 
发送时间: 2016年3月31日 19:55
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [Heat] Re-evaluate conditions specification

On Thu, Mar 31, 2016 at 10:40 AM, Thomas Herve  wrote:
> Hi all,
>
> As the patches for conditions support are incoming, I've found 
> something in the code (and the spec) I'm not really happy with. We're 
> creating a new top-level section in the template called "conditions"
> which holds names that can be reused for conditionally creating 
> resource.
>
> While it's fine and maps to what AWS does, I think it's a bit 
> short-sighted and limited. What I have suggested in the past is to 
> have a "variables" (or whatever you want to call it) section, where 
> one can declare names and values. Then we can add an intrinsic 
> function to retrieve data from there, and use that for examples for 
> conditions.

I was asked to give examples, here's at least one that can illustrate what I 
meant:

parameters:
   host:
  type: string
   port:
  type: string

variables:
   endpoint:
  str_replace:
template:
   http://HOST:PORT/
params:
   HOST: {get_param: host}
   PORT: {get_param: port}

resources:
   config1:
  type: OS::Heat::StructuredConfig
  properties:
config:
   hosts: [{get_variable: endpoint}]

--
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev