Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Thomas Spatzier
 From: Mike Spreitzer mspre...@us.ibm.com
 To: OpenStack Development Mailing List \(not for usage questions\)
 openstack-dev@lists.openstack.org
 Date: 03/04/2014 07:10
 Subject: Re: [openstack-dev] [heat] metadata for a HOT

 Zane Bitter zbit...@redhat.com wrote on 04/02/2014 05:36:43 PM:

  I think that if you're going to propose a new feature, you should at
  least give us a clue who you think is going to use it and what for ;)

 I was not eager to do that yet because I have not found a fully
 satisfactory answer yet, at this point I am exploring options.  But
 the problem I am thinking about is how Heat might connect to a
 holistic scheduler (a scheduler that makes a joint decision about a
 bunch of resources of various types).  Such a scheduler needs input
 describing the things to be scheduled and the policies to apply in
 scheduling; the first half of that sounds a lot like a Heat
 template, so my thoughts go in that direction.  But the HOT language
 today (since https://review.openstack.org/#/c/83758/ was merged)
 does not have a place to put policy that is not specific to a
singleresource.

I think you bring up a specific use case here, i.e. applying policies for
placement/scheduling when deploying a stack. This is just a thought, but I
wonder whether it would make more sense to then define a specific extension
to HOT instead of having a generic metadata section and stuffing everything
that does not fit into other places into metadata.

I mean, the use case Keith brought up are completely different (UI and user
related), and I understand both use cases. But is the idea to put just
everything into metadata, or would different classes of use cases justify
different section? The latter would enforce better documentation of
semantics. If everyhing goes into a metadata section, the contents also
need to be clearly specified. Otherwise, the resulting template won't be
portable. Ok, the standard HOT stuff will be portable, but not the
metadata, so no two users will be able to interpret it the same way.


  IIRC this has been discussed in the past and the justifications for
  including it in the template (as opposed to allowing metadata to be
  attached in the ReST API, as other projects already do for many things)

  were not compelling.

 I see that Keith Bray mentioned https://wiki.openstack.org/wiki/
 Heat/StackMetadata and https://wiki.openstack.org/wiki/Heat/UI in
 another reply on this thread.  Are there additional places to look
 to find that discussion?

 I have also heard that there has been discussion of language
 extension issues.  Is that a separate discussion and, if so, where
 can I read it?

 Thanks,
 Mike
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Steven Dake

On 04/02/2014 08:41 PM, Keith Bray wrote:

https://wiki.openstack.org/wiki/Heat/StackMetadata

https://wiki.openstack.org/wiki/Heat/UI

-Keith


Keith,

Taking a look at the UI specification, I thought I'd take a look at 
adding parameter grouping and ordering to the hot_spec.rst file. That 
seems like a really nice constrained use case with a clear way to 
validate that folks aren't adding magic to the template for their custom 
environments.  During that, I noticed it is is already implemented.


What is nice about this specific use case is it is something that can be 
validated by the parser.  For example, the parser could enforce that 
parameters in the parameter-groups section actually exist as parameters 
in the parameters section.  Essentially this particular use case 
*enforces* good heat template implementation without an opportunity for 
HOT template developers to jam customized data blobs into the template.


Stack keywords on the other hand doesn't necessarily follow this model.  
I understand the use case, but it would be possible to jam unstructured 
metadata into the template.  That said, the limitations on the jamming 
custom metadata are one deep and it has a clear use case (categorization 
of templates for support/UI rendering purposes).


I could be wrong, but I think the aversion to a general metadata section 
is centered around the problem of different people doing different 
things in a non-standardized way.


I think if we were to revisit the metadata proposal, one thing that 
might lead to a more successful outcome is actually defining what goes 
in the metadata, rather then allowing the metadata to be completely 
free-form as the HOT developer sees fit to implement it.


For example just taking the keywords proposal:
metadata:
  composed_of:
  - wordpress
  - mysql
  architecture:
  - lamp

Even though this metadata can't necessarily be validated, it can be 
documented.  I definitely have a -2 aversion to free-form metadata 
structuring, and am +0 on allowing the information to be declared in a 
non-validated way.


I don't believe the idea of structured metadata based upon real use 
cases has really been explored or -2'ed.


Regards,
-steve


From: Lingxian Kong anlin.k...@gmail.com mailto:anlin.k...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage 
questions) openstack-dev@lists.openstack.org 
mailto:openstack-dev@lists.openstack.org

Date: Wednesday, April 2, 2014 9:31 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org 
mailto:openstack-dev@lists.openstack.org

Subject: Re: [openstack-dev] [heat] metadata for a HOT

Is there any relevant wiki or specification doc?


2014-04-03 4:45 GMT+08:00 Mike Spreitzer mspre...@us.ibm.com
mailto:mspre...@us.ibm.com:

I would like to suggest that a metadata section be allowed at
the top level of a HOT.  Note that while resources in a stack
can have metadata, there is no way to put metadata on a stack
itself.  What do you think?

Thanks,
Mike
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
*---*

*Lingxian Kong*
Huawei Technologies Co.,LTD.
IT Product Line CloudOS PDU
China, Xi'an
Mobile: +86-18602962792
Email: konglingx...@huawei.com mailto:konglingx...@huawei.com;
anlin.k...@gmail.com mailto:anlin.k...@gmail.com



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Clint Byrum
Excerpts from Thomas Spatzier's message of 2014-04-03 08:36:20 -0700:
  From: Mike Spreitzer mspre...@us.ibm.com
  To: OpenStack Development Mailing List \(not for usage questions\)
  openstack-dev@lists.openstack.org
  Date: 03/04/2014 07:10
  Subject: Re: [openstack-dev] [heat] metadata for a HOT
 
  Zane Bitter zbit...@redhat.com wrote on 04/02/2014 05:36:43 PM:
 
   I think that if you're going to propose a new feature, you should at
   least give us a clue who you think is going to use it and what for ;)
 
  I was not eager to do that yet because I have not found a fully
  satisfactory answer yet, at this point I am exploring options.  But
  the problem I am thinking about is how Heat might connect to a
  holistic scheduler (a scheduler that makes a joint decision about a
  bunch of resources of various types).  Such a scheduler needs input
  describing the things to be scheduled and the policies to apply in
  scheduling; the first half of that sounds a lot like a Heat
  template, so my thoughts go in that direction.  But the HOT language
  today (since https://review.openstack.org/#/c/83758/ was merged)
  does not have a place to put policy that is not specific to a
 singleresource.
 
 I think you bring up a specific use case here, i.e. applying policies for
 placement/scheduling when deploying a stack. This is just a thought, but I
 wonder whether it would make more sense to then define a specific extension
 to HOT instead of having a generic metadata section and stuffing everything
 that does not fit into other places into metadata.
 

Ever read about Larry no modes Tesler? Read up on his arguments
against modes.

I would much prefer any policies to be actual resources which the
resources interact with, rather than template wide modes.

 I mean, the use case Keith brought up are completely different (UI and user
 related), and I understand both use cases. But is the idea to put just
 everything into metadata, or would different classes of use cases justify
 different section? The latter would enforce better documentation of
 semantics. If everyhing goes into a metadata section, the contents also
 need to be clearly specified. Otherwise, the resulting template won't be
 portable. Ok, the standard HOT stuff will be portable, but not the
 metadata, so no two users will be able to interpret it the same way.


We had a fairly long debate about keywords and meta-information in HOT
and I thought we came to the conclusion that it belongs in the API and
not in the template language.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Clint Byrum
Excerpts from Mike Spreitzer's message of 2014-04-02 22:10:21 -0700:
 Zane Bitter zbit...@redhat.com wrote on 04/02/2014 05:36:43 PM:
 
  I think that if you're going to propose a new feature, you should at 
  least give us a clue who you think is going to use it and what for ;)
 
 I was not eager to do that yet because I have not found a fully 
 satisfactory answer yet, at this point I am exploring options.  But the 
 problem I am thinking about is how Heat might connect to a holistic 
 scheduler (a scheduler that makes a joint decision about a bunch of 
 resources of various types).  Such a scheduler needs input describing the 
 things to be scheduled and the policies to apply in scheduling; the first 
 half of that sounds a lot like a Heat template, so my thoughts go in that 
 direction.  But the HOT language today (since 
 https://review.openstack.org/#/c/83758/ was merged) does not have a place 
 to put policy that is not specific to a single resource.
 

Things that affect the stack as a whole really belong in the stack
API. That would also put them in the OS::Heat::Stack resource, so the
template language already supports that.

As for policies which might affect a holistic scheduler, those can just
be resources as well. Just like deployments relate to servers, resources
can relate to any policies they need.

  IIRC this has been discussed in the past and the justifications for 
  including it in the template (as opposed to allowing metadata to be 
  attached in the ReST API, as other projects already do for many things) 
  were not compelling.
 
 I see that Keith Bray mentioned 
 https://wiki.openstack.org/wiki/Heat/StackMetadata and 
 https://wiki.openstack.org/wiki/Heat/UI in another reply on this thread. 
 Are there additional places to look to find that discussion?
 
 I have also heard that there has been discussion of language extension 
 issues.  Is that a separate discussion and, if so, where can I read it?

I would prefer that we focus on making HOT composable rather than
extensible. If there is actually something missing from the root of the
language, then it should be in the language. Use cases should almost
always try to find a way to work as resources first, and then if that
is unwieldy, look into language enhancements to factor things out.

I think the way hot-software-config has taken shape is a prime example
of this. We took the most common patterns and made a set of resources
that encapsulate them. But we didn't have to extend the language any. It
is all done in resources. (Kudos to Steve Baker for getting it done btw,
this was _not_ a small amount of work).

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Keith Bray
Steve, agreed.  Your description I believe is the conclusion that the community 
came to when this was perviously discussed, and we managed to get the 
implementation of parameter grouping and ordering [1] that you mentioned which 
has been very helpful.  I don't think we landed the keywords blueprint [2], 
which may be controversial because it is essentially unstructured. I wanted to 
make sure Mike had the links for historical context, but certainly understand 
and appreciate your point of view here.  I wasn't able to find the email 
threads to point Mike to, but assume they exist in the list archives somewhere.

We proposed another specific piece of template data [3] which I can't remember 
whether it was met with resistance or we just didn't get to implementing it 
since we knew we would have to store other data specific to our uses cases in 
other files anyway.   We decided to go with storing our extra information in a 
catalog (really just a Git repo with a README.MD [4]) for now  until we can 
implement acceptable catalog functionality somewhere like Glance, hopefully in 
the Juno cycle.  When we want to share the template, we share all the files in 
the repo (inclusive of the README.MD).  It would be more ideal if we could 
share a single file (package) inclusive of the template and corresponding help 
text and any other UI hint info that would helpful.  I expect service providers 
to have differing views of the extra data they want to store with a template... 
So it'd just be nice to have a way to account for service providers to store 
their unique data along with a template that is easy to share and is part of 
the template package.  We bring up portability and structured data often, but 
I'm starting to realize that portability of a template breaks down unless every 
service provider runs exactly the same Heat resources, same image IDs, flavor 
types, etc.). I'd like to drive more standardization of data for image and 
template data into Glance so that in HOT we can just declare things like 
Linux, Flavor Ubuntu, latest LTS, minimum 1Gig and automatically discover and 
choose the right image to provision, or error if a suitable match can not be 
found.  The Murano team has been hinting at wanting to solve a similar problem, 
but with a broader vision from a complex-multi application declaration 
perspective that crosses multiple templates or is a layer above just matching 
to what capabilities Heat resources provide and matching against capabilities 
that a catalog of templates provide (and mix that with capabilities the cloud 
API services provide).  I'm not yet convinced that can't be done with a parent 
Heat template since we already have the declarative constructs and language 
well defined, but I appreciate the use case and perspective those folks are 
bringing to the conversation.

[1] https://blueprints.launchpad.net/heat/+spec/parameter-grouping-ordering
 https://wiki.openstack.org/wiki/Heat/UI#Parameter_Grouping_and_Ordering

[2] https://blueprints.launchpad.net/heat/+spec/stack-keywords
https://wiki.openstack.org/wiki/Heat/UI#Stack_Keywords

[3] https://blueprints.launchpad.net/heat/+spec/add-help-text-to-template
https://wiki.openstack.org/wiki/Heat/UI#Help_Text

[4] Ex. Help Text accompanying a template in README.MD format:
https://github.com/rackspace-orchestration-templates/docker

-Keith

From: Steven Dake sd...@redhat.commailto:sd...@redhat.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Thursday, April 3, 2014 10:30 AM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [heat] metadata for a HOT

On 04/02/2014 08:41 PM, Keith Bray wrote:
https://wiki.openstack.org/wiki/Heat/StackMetadata

https://wiki.openstack.org/wiki/Heat/UI

-Keith

Keith,

Taking a look at the UI specification, I thought I'd take a look at adding 
parameter grouping and ordering to the hot_spec.rst file.  That seems like a 
really nice constrained use case with a clear way to validate that folks aren't 
adding magic to the template for their custom environments.  During that, I 
noticed it is is already implemented.

What is nice about this specific use case is it is something that can be 
validated by the parser.  For example, the parser could enforce that parameters 
in the parameter-groups section actually exist as parameters in the parameters 
section.  Essentially this particular use case *enforces* good heat template 
implementation without an opportunity for HOT template developers to jam 
customized data blobs into the template.

Stack keywords on the other hand doesn't necessarily follow this model.  I 
understand the use case, but it would be possible to jam unstructured metadata 
into the template.  That said, the limitations on the jamming custom metadata 
are one deep

Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Mike Spreitzer
Clint Byrum cl...@fewbar.com wrote on 04/03/2014 01:10:30 PM:

 Things that affect the stack as a whole really belong in the stack
 API. That would also put them in the OS::Heat::Stack resource, so the
 template language already supports that.

The OS::Heat::Stack resource is one of several that create nested stacks;
we should be able to apply holistic scheduling to all stacks, regardless
of whether they are nest or which kind of nested stack they are.
Yes, if holistic scheduling were a feature in the Heat API then all kinds 
of
resources that create nested stacks should expose that feature
(shout out to Trove, autoscaling groups, ...).

 As for policies which might affect a holistic scheduler, those can just
 be resources as well. Just like deployments relate to servers, resources
 can relate to any policies they need.

A holistic scheduler needs input that describes all the resources to be 
scheduled as well as all the policies that apply.  Should a template 
contain a resource whose input includes a copy of the rest of the 
template?

 I would prefer that we focus on making HOT composable rather than
 extensible. If there is actually something missing from the root of the
 language, then it should be in the language. Use cases should almost
 always try to find a way to work as resources first, and then if that
 is unwieldy, look into language enhancements to factor things out.

Yeah, I would too.  Like I said, I have no satisfactory solution yet. Here 
is more of the problem.  I would like to follow an evolutionary path 
starting from the instance groups that are in Nova today.  I think I can 
outline such an evolution.  I am sure there will be debate about it.  I am 
even more sure that it will take time to accomplish that evolution.  OTOH, 
locally we have a holistic scheduler already working.  We want to be able 
to start using it today.  What can we do in this interim, and how can we 
arrange things to do progressive convergence so that the interim solution 
evolves as Nova  scheduling evolve, so that there is no big switch at the 
end to the end solution?

Thanks,
Mike___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Mike Spreitzer
Keith Bray keith.b...@rackspace.com wrote on 04/03/2014 01:50:28 PM:

 We proposed another specific piece of template data [3] which I 
 can't remember whether it was met with resistance or we just didn't 
 get to implementing it since we knew we would have to store other 
 data specific to our uses cases in other files anyway.   We decided 
 to go with storing our extra information in a catalog (really just a
 Git repo with a README.MD [4]) for now  until we can implement 
 acceptable catalog functionality somewhere like Glance, hopefully in
 the Juno cycle.  When we want to share the template, we share all 
 the files in the repo (inclusive of the README.MD).  It would be 
 more ideal if we could share a single file (package) inclusive of 
 the template and corresponding help text and any other UI hint info 
 that would helpful.  I expect service providers to have differing 
 views of the extra data they want to store with a template... So 
 it'd just be nice to have a way to account for service providers to 
 store their unique data along with a template that is easy to share 
 and is part of the template package.  We bring up portability and 
 structured data often, but I'm starting to realize that portability 
 of a template breaks down unless every service provider runs exactly
 the same Heat resources, same image IDs, flavor types, etc.). I'd 
 like to drive more standardization of data for image and template 
 data into Glance so that in HOT we can just declare things like 
 Linux, Flavor Ubuntu, latest LTS, minimum 1Gig and automatically 
 discover and choose the right image to provision, or error if a 
 suitable match can not be found.  The Murano team has been hinting 
 at wanting to solve a similar problem, but with a broader vision 
 from a complex-multi application declaration perspective that 
 crosses multiple templates or is a layer above just matching to what
 capabilities Heat resources provide and matching against 
 capabilities that a catalog of templates provide (and mix that with 
 capabilities the cloud API services provide).  I'm not yet convinced
 that can't be done with a parent Heat template since we already have
 the declarative constructs and language well defined, but I 
 appreciate the use case and perspective those folks are bringing to 
 the conversation.

Keith, thanks for the background and wider context.  I am responding 
directly on my original point elsewhere, but let me pick up on a couple of 
things you mentioned in your wider context.  I definitely see a reason for 
interest in packaging something bigger than a single template.  As one 
very simple example, I have been exercising OS::Heat::AutoScalingGroup 
with a pair of templates (an outer template in which the ASG is a 
resource, where the element being scaled is a nested stack, prescribed by 
the other template in my package).  Since we are so fond here of solving 
problems with nested templates, I think there will be an increasing need 
to package together not only templates but also environment snippets (and, 
yeah, we need to smooth out how the receiver combines environment 
snippets).

I agree that template portability is problematic due to the 
non-portability of image UUIDs and flavors.  The approach you pointed 
towards looks attractive, but it is challenging to enable a template 
author to write a specification that is not too precise and not too 
liberal --- particularly since the template author has a hard time 
anticipating the creativity with which the receiving environment is 
populated.  I assume this has been extensively discussed already.  If not 
I may make some noise later.

Thanks,
Mike___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Thomas Spatzier
 From: Keith Bray keith.b...@rackspace.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: 03/04/2014 19:51
 Subject: Re: [openstack-dev] [heat] metadata for a HOT

 Steve, agreed.  Your description I believe is the conclusion that
 the community came to when this was perviously discussed, and we
 managed to get the implementation of parameter grouping and ordering
 [1] that you mentioned which has been very helpful.  I don't think
 we landed the keywords blueprint [2], which may be controversial
 because it is essentially unstructured. I wanted to make sure Mike
 had the links for historical context, but certainly understand and
 appreciate your point of view here.  I wasn't able to find the email
 threads to point Mike to, but assume they exist in the list archives
 somewhere.

 We proposed another specific piece of template data [3] which I
 can't remember whether it was met with resistance or we just didn't
 get to implementing it since we knew we would have to store other
 data specific to our uses cases in other files anyway.   We decided
 to go with storing our extra information in a catalog (really just a
 Git repo with a README.MD [4]) for now  until we can implement
 acceptable catalog functionality somewhere like Glance, hopefully in
 the Juno cycle.  When we want to share the template, we share all
 the files in the repo (inclusive of the README.MD).  It would be
 more ideal if we could share a single file (package) inclusive of
 the template and corresponding help text and any other UI hint info
 that would helpful.  I expect service providers to have differing

I agree that packaging all stuff that makes up a template (which will in
many cases not be a single template file, but nested templates,
environments, scripts, ...) in one archive. We have this concept in TOSCA
and I am sure we will have to implement a solution for this as part of the
TOSCA YAML to HOT converter work that we started. If several people see
this requirement, let's see if we can join forces on a common solution.

 views of the extra data they want to store with a template... So
 it'd just be nice to have a way to account for service providers to
 store their unique data along with a template that is easy to share
 and is part of the template package.  We bring up portability and
 structured data often, but I'm starting to realize that portability
 of a template breaks down unless every service provider runs exactly
 the same Heat resources, same image IDs, flavor types, etc.). I'd
 like to drive more standardization of data for image and template
 data into Glance so that in HOT we can just declare things like
 Linux, Flavor Ubuntu, latest LTS, minimum 1Gig and automatically
 discover and choose the right image to provision, or error if a
 suitable match can not be found.  The Murano team has been hinting

Sahdev from our team recently created a BP for exactly that scenario.
Please have a look and see if that is in line with your thinking and
provide comments as necessary:

https://blueprints.launchpad.net/heat/+spec/constraint-based-flavors-and-images

 at wanting to solve a similar problem, but with a broader vision
 from a complex-multi application declaration perspective that
 crosses multiple templates or is a layer above just matching to what
 capabilities Heat resources provide and matching against
 capabilities that a catalog of templates provide (and mix that with
 capabilities the cloud API services provide).  I'm not yet convinced
 that can't be done with a parent Heat template since we already have
 the declarative constructs and language well defined, but I
 appreciate the use case and perspective those folks are bringing to
 the conversation.

 [1]
https://blueprints.launchpad.net/heat/+spec/parameter-grouping-ordering
  https://wiki.openstack.org/wiki/Heat/UI#Parameter_Grouping_and_Ordering

 [2] https://blueprints.launchpad.net/heat/+spec/stack-keywords
 https://wiki.openstack.org/wiki/Heat/UI#Stack_Keywords

 [3] https://blueprints.launchpad.net/heat/+spec/add-help-text-to-template
 https://wiki.openstack.org/wiki/Heat/UI#Help_Text

 [4] Ex. Help Text accompanying a template in README.MD format:
 https://github.com/rackspace-orchestration-templates/docker

 -Keith

 From: Steven Dake sd...@redhat.com
 Reply-To: OpenStack Development Mailing List (not for usage questions)

 openstack-dev@lists.openstack.org
 Date: Thursday, April 3, 2014 10:30 AM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [heat] metadata for a HOT

 On 04/02/2014 08:41 PM, Keith Bray wrote:
 https://wiki.openstack.org/wiki/Heat/StackMetadata

 https://wiki.openstack.org/wiki/Heat/UI

 -Keith

 Keith,

 Taking a look at the UI specification, I thought I'd take a look at
 adding parameter grouping and ordering to the hot_spec.rst file.
 That seems like a really nice constrained use case

Re: [openstack-dev] [heat] metadata for a HOT

2014-04-03 Thread Mark Washenberger
On Thu, Apr 3, 2014 at 10:50 AM, Keith Bray keith.b...@rackspace.comwrote:

  Steve, agreed.  Your description I believe is the conclusion that the
 community came to when this was perviously discussed, and we managed to get
 the implementation of parameter grouping and ordering [1] that you
 mentioned which has been very helpful.  I don't think we landed the
 keywords blueprint [2], which may be controversial because it is
 essentially unstructured. I wanted to make sure Mike had the links for
 historical context, but certainly understand and appreciate your point of
 view here.  I wasn't able to find the email threads to point Mike to, but
 assume they exist in the list archives somewhere.

  We proposed another specific piece of template data [3] which I can't
 remember whether it was met with resistance or we just didn't get to
 implementing it since we knew we would have to store other data specific to
 our uses cases in other files anyway.   We decided to go with storing our
 extra information in a catalog (really just a Git repo with a README.MD[4]) 
 for now  until we can implement acceptable catalog functionality
 somewhere like Glance, hopefully in the Juno cycle.  When we want to share
 the template, we share all the files in the repo (inclusive of the
 README.MD).  It would be more ideal if we could share a single file
 (package) inclusive of the template and corresponding help text and any
 other UI hint info that would helpful.  I expect service providers to have
 differing views of the extra data they want to store with a template... So
 it'd just be nice to have a way to account for service providers to store
 their unique data along with a template that is easy to share and is part
 of the template package.  We bring up portability and structured data
 often, but I'm starting to realize that portability of a template breaks
 down unless every service provider runs exactly the same Heat resources,
 same image IDs, flavor types, etc.). I'd like to drive more standardization
 of data for image and template data into Glance so that in HOT we can just
 declare things like Linux, Flavor Ubuntu, latest LTS, minimum 1Gig and
 automatically discover and choose the right image to provision, or error if
 a suitable match can not be found.


Yes, this is exactly the use case that has been driving our consideration
of the artifacts resource in Glance.

You mentioned discovery of compatible resources. I think its an important
use case, but I think the export and import approach can also be very
useful and I'd like to believe it is the general solution to cloud
portability.


  The Murano team has been hinting at wanting to solve a similar problem,
 but with a broader vision from a complex-multi application declaration
 perspective that crosses multiple templates or is a layer above just
 matching to what capabilities Heat resources provide and matching against
 capabilities that a catalog of templates provide (and mix that with
 capabilities the cloud API services provide).  I'm not yet convinced that
 can't be done with a parent Heat template since we already have the
 declarative constructs and language well defined, but I appreciate the use
 case and perspective those folks are bringing to the conversation.

  [1]
 https://blueprints.launchpad.net/heat/+spec/parameter-grouping-ordering
  https://wiki.openstack.org/wiki/Heat/UI#Parameter_Grouping_and_Ordering

  [2] https://blueprints.launchpad.net/heat/+spec/stack-keywords
 https://wiki.openstack.org/wiki/Heat/UI#Stack_Keywords

  [3] https://blueprints.launchpad.net/heat/+spec/add-help-text-to-template
 https://wiki.openstack.org/wiki/Heat/UI#Help_Text

  [4] Ex. Help Text accompanying a template in README.MD format:
 https://github.com/rackspace-orchestration-templates/docker

  -Keith

   From: Steven Dake sd...@redhat.com

 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Thursday, April 3, 2014 10:30 AM

 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [heat] metadata for a HOT

   On 04/02/2014 08:41 PM, Keith Bray wrote:

 https://wiki.openstack.org/wiki/Heat/StackMetadata

  https://wiki.openstack.org/wiki/Heat/UI

  -Keith

  Keith,

 Taking a look at the UI specification, I thought I'd take a look at adding
 parameter grouping and ordering to the hot_spec.rst file.  That seems like
 a really nice constrained use case with a clear way to validate that folks
 aren't adding magic to the template for their custom environments.  During
 that, I noticed it is is already implemented.

 What is nice about this specific use case is it is something that can be
 validated by the parser.  For example, the parser could enforce that
 parameters in the parameter-groups section actually exist as parameters in
 the parameters section.  Essentially this particular use case *enforces*
 good heat template

Re: [openstack-dev] [heat] metadata for a HOT

2014-04-02 Thread Zane Bitter

On 02/04/14 16:45, Mike Spreitzer wrote:

I would like to suggest that a metadata section be allowed at the top
level of a HOT.  Note that while resources in a stack can have metadata,
there is no way to put metadata on a stack itself.  What do you think?


I think that if you're going to propose a new feature, you should at 
least give us a clue who you think is going to use it and what for ;)


IIRC this has been discussed in the past and the justifications for 
including it in the template (as opposed to allowing metadata to be 
attached in the ReST API, as other projects already do for many things) 
were not compelling.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] metadata for a HOT

2014-04-02 Thread Mike Spreitzer
Zane Bitter zbit...@redhat.com wrote on 04/02/2014 05:36:43 PM:

 I think that if you're going to propose a new feature, you should at 
 least give us a clue who you think is going to use it and what for ;)

I was not eager to do that yet because I have not found a fully 
satisfactory answer yet, at this point I am exploring options.  But the 
problem I am thinking about is how Heat might connect to a holistic 
scheduler (a scheduler that makes a joint decision about a bunch of 
resources of various types).  Such a scheduler needs input describing the 
things to be scheduled and the policies to apply in scheduling; the first 
half of that sounds a lot like a Heat template, so my thoughts go in that 
direction.  But the HOT language today (since 
https://review.openstack.org/#/c/83758/ was merged) does not have a place 
to put policy that is not specific to a single resource.

 IIRC this has been discussed in the past and the justifications for 
 including it in the template (as opposed to allowing metadata to be 
 attached in the ReST API, as other projects already do for many things) 
 were not compelling.

I see that Keith Bray mentioned 
https://wiki.openstack.org/wiki/Heat/StackMetadata and 
https://wiki.openstack.org/wiki/Heat/UI in another reply on this thread. 
Are there additional places to look to find that discussion?

I have also heard that there has been discussion of language extension 
issues.  Is that a separate discussion and, if so, where can I read it?

Thanks,
Mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev