Re: [openstack-dev] [heat][tripleo] Heat memory usage in the TripleO gate during Ocata

[Hugh Brock]

Why would TripleO not move to convergence at the earliest possible point?

On Jan 6, 2017 10:37 PM, "Thomas Herve"  wrote:

> On Fri, Jan 6, 2017 at 6:12 PM, Zane Bitter  wrote:
> > tl;dr everything looks great, and memory usage has dropped by about 64%
> > since the initial Newton release of Heat.
> >
> > I re-ran my analysis of Heat memory usage in the tripleo-heat-templates
> > gate. (This is based on the gate-tripleo-ci-centos-7-ovb-nonha job.)
> Here's
> > a pretty picture:
> >
> > https://fedorapeople.org/~zaneb/tripleo-memory/20170105/heat_memused.png
> >
> > There is one major caveat here: for the period marked in grey where it
> says
> > "Only 2 engine workers", the job was configured to use only 2
> heat-enginer
> > worker processes instead of 4, so this is not an apples-to-apples
> > comparison. The inital drop at the beginning and the subsequent bounce at
> > the end are artifacts of this change. Note that the stable/newton branch
> is
> > _still_ using only 2 engine workers.
> >
> > The rapidly increasing usage on the left is due to increases in the
> > complexity of the templates during the Newton cycle. It's clear that if
> > there has been any similar complexity growth during Ocata, it has had a
> tiny
> > effect on memory consumption in comparison.
>
> Thanks a lot for the analysis. It's great that things haven't gotten off
> track.
>
> > I tracked down most of the step changes to identifiable patches:
> >
> > 2016-10-07: 2.44GiB -> 1.64GiB
> >  - https://review.openstack.org/382068/ merged, making ResourceInfo
> classes
> > more memory-efficient. Judging by the stable branch (where this and the
> > following patch were merged at different times), this was responsible for
> > dropping the memory usage from 2.44GiB -> 1.83GiB. (Which seems like a
> > disproportionately large change?)
>
> Without wanting to get the credit, I believe
> https://review.openstack.org/377061/ is more likely the reason here.
>
> >  - https://review.openstack.org/#/c/382377/ merged, so we no longer
> create
> > multiple yaql contexts. (This was responsible for the drop from 1.83GiB
> ->
> > 1.64GiB.)
> >
> > 2016-10-17: 1.62GiB -> 0.93GiB
> >  - https://review.openstack.org/#/c/386696/ merged, reducing the number
> of
> > engine workers on the undercloud to 2.
> >
> > 2016-10-19: 0.93GiB -> 0.73GiB (variance also seemed to drop after this)
> >  - https://review.openstack.org/#/c/386247/ merged (on 2016-10-16),
> avoiding
> > loading all nested stacks in a single process simultaneously much of the
> > time.
> >  - https://review.openstack.org/#/c/383839/ merged (on 2016-10-16),
> > switching output calculations to RPC to avoid almost all simultaneous
> > loading of all nested stacks.
> >
> > 2016-11-08: 0.76GiB -> 0.70GiB
> >  - This one is a bit of a mystery???
>
> Possibly https://review.openstack.org/390064/ ? Reducing the
> environment size could have an effect.
>
> > 2016-11-22: 0.69GiB -> 0.50GiB
> >  - https://review.openstack.org/#/c/398476/ merged, improving the
> efficiency
> > of resource listing?
> >
> > 2016-12-01: 0.49GiB -> 0.88GiB
> >  - https://review.openstack.org/#/c/399619/ merged, returning the
> number of
> > engine workers on the undercloud to 4.
> >
> > It's not an exact science because IIUC there's a delay between a patch
> > merging in Heat and it being used in subsequent t-h-t gate jobs. e.g. the
> > change to getting outputs over RPC landed the day before the
> > instack-undercloud patch that cut the number of engine workers, but the
> > effects don't show up until 2 days after. I'd love to figure out what
> > happened on the 8th of November, but I can't correlate it to anything
> > obvious. The attribution of the change on the 22nd also seems dubious,
> but
> > the timing adds up (including on stable/newton).
> >
> > It's fair to say that none of the other patches we merged in an attempt
> to
> > reduce memory usage had any discernible effect :D
> >
> > It's worth reiterating that TripleO still disables convergence in the
> > undercloud, so these are all tests of the legacy code path. It would be
> > great if we could set up a non-voting job on t-h-t with convergence
> enabled
> > and start tracking memory use over time there too. As a first step,
> maybe we
> > could at least add an experimental job on Heat to give us a baseline?
>
> +1. We haven't made any huge changes into that direction, but having
> some info would be great.
>
> --
> Thomas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [TripleO][UI] Port number for frontend app

[Hugh Brock]

On Wed, Aug 24, 2016 at 12:18 PM, Dmitry Tantsur <dtant...@redhat.com> wrote:
> On 08/22/2016 08:08 AM, Honza Pokorny wrote:
>>
>> Hello folks,
>>
>> We've been using port 3000 for the GUI during development and testing.
>> Now that we're working on packaging and shipping our code, we're
>> wondering if port 3000 is still the best choice.
>>
>> Would 3000 conflict with any other services?  Is there a better option?
>
>
> I think the best option is to run it as wsgi in the same Apache instance as
> e.g. Horizon (and I guess Keystone and other services in the future as
> well), just with a different leading path. Not sure how doable it is though.
>

I am sure that that is the Correct Way to do it. We don't want another
process listening on a different port after we've put all this effort
into getting everything in WSGI. Is that doable in the time we have
remaining?

--Hugh

-- 
  ,   ,| Hugh Brock, hbr...@redhat.com
  )-_"""_-(| Director of Engineering, OpenStack Management
 ./ o\ /o \.   | TripleO: Install, configure, and scale OpenStack.
. \__/ \__/ .  | http://rdoproject.org, http://tripleo.org
...   V   ...  |
... - - - ...  | "I know that you believe you understand what you
 .   - -   .   | think I said, but I'm not sure you realize that what
  `-.-´| you heard is not what I meant." --Robert McCloskey
 "TripleOwl"

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][CI] Memory shortage in HA jobs, please increase it

[Hugh Brock]

On Fri, Aug 19, 2016 at 11:41 AM, Derek Higgins <der...@redhat.com> wrote:
> On 19 August 2016 at 00:07, Sagi Shnaidman <sshna...@redhat.com> wrote:
>> Hi,
>>
>> we have a problem again with not enough memory in HA jobs, all of them
>> constantly fails in CI: http://status-tripleoci.rhcloud.com/
>
> Have we any idea why we need more memory all of a sudden? For months
> the overcloud nodes have had 5G of RAM, then last week[1] we bumped it
> too 5.5G now we need it bumped too 6G.
>
> If a new service has been added that is needed on the overcloud then
> bumping to 6G is expected and probably the correct answer but I'd like
> to see us avoiding blindly increasing the resources each time we see
> out of memory errors without investigating if there was a regression
> causing something to start hogging memory.
>
> Sorry if it seems like I'm being picky about this (I seem to resist
> these bumps every time they come up) but there are two good reasons to
> avoid this if possible
> o at peak we are currently configured to run 75 simultaneous jobs
> (although we probably don't reach that at the moment), and each HA job
> has 5 baremetal nodes so bumping from 5G too 6G increases the amount
> of RAM ci can use at peak by 375G
> o When we bump the RAM usage of baremetal nodes from 5G too 6G what
> we're actually doing is increasing the minimum requirements for
> developers from 28G(or whatever the number is now) too 32G
>
> So before we bump the number can we just check first if its justified,
> as I've watched this number increase from 2G since we started running
> tripleo-ci
>
> thanks,
> Derek.
>
> [1] - https://review.openstack.org/#/c/353655/

Wondering if it makes sense to enable any but the most basic overcloud
services in TripleO CI. The idea of using some type of on-demand job
for services other than the ones needed for the ping test has been
proposed elsewhere -- maybe this should be our default mode for
TripleO CI. Thoughts?

--Hugh


>> I've created a patch that will increase it[1], but we need to increase it
>> right now on rh1.
>> I can't do it now, because unfortunately I'll not be able to watch this if
>> it works and no problems appear.
>> TripleO CI cloud admins, please increase the memory for baremetal flavor on
>> rh1 tomorrow (to 6144?).
>>
>> Thanks
>>
>> [1] https://review.openstack.org/#/c/357532/
>> --
>> Best regards
>> Sagi Shnaidman
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
  ,   ,| Hugh Brock, hbr...@redhat.com
  )-_"""_-(| Director of Engineering, OpenStack Management
 ./ o\ /o \.   | TripleO: Install, configure, and scale OpenStack.
. \__/ \__/ .  | http://rdoproject.org, http://tripleo.org
...   V   ...  |
... - - - ...  | "I know that you believe you understand what you
 .   - -   .   | think I said, but I'm not sure you realize that what
  `-.-´| you heard is not what I meant." --Robert McCloskey
 "TripleOwl"

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [E] [TripleO] scripts to do post deployment analysis of an overcloud

[Hugh Brock]

On Mon, Aug 8, 2016 at 4:02 PM, Tomas Sedovic <tsedo...@redhat.com> wrote:
> On 08/08/2016 11:05 AM, Hugh Brock wrote:
>>
>> On Wed, Aug 3, 2016 at 4:49 PM, Joe Talerico <jtale...@redhat.com> wrote:
>>>
>>> On Wed, Jul 27, 2016 at 2:04 AM, Hugh Brock <hbr...@redhat.com> wrote:
>>>>
>>>> On Jul 26, 2016 8:08 PM, "Gordon, Kent"
>>>> <kent.gor...@verizonwireless.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> -Original Message-
>>>>>> From: Gonéri Le Bouder [mailto:gon...@lebouder.net]
>>>>>> Sent: Tuesday, July 26, 2016 12:24 PM
>>>>>> To: openstack-dev@lists.openstack.org
>>>>>> Subject: [E] [openstack-dev] [TripleO] scripts to do post deployment
>>>>>> analysis
>>>>>> of an overcloud
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> For the Distributed-CI[0] project, we did two scripts[1] that we use
>>>>>> to
>>>>>> extract
>>>>>
>>>>>
>>>>> Links not included in message
>>>>>
>>>>>> information from an overcloud.
>>>>>> We use this information to improve the readability of the deployment
>>>>>> logs.
>>>>>> I attached an example to show how we use the extracted stack
>>>>>> information.
>>>>>>
>>>>>> Now my question, do you know some other tools that we can use to do
>>>>>> this
>>>>>> kind of anaylsis?
>>>>>> --
>>>>>> Gonéri Le Bouder
>>>>>
>>>>>
> 
>>
>>
>> Tomaš, this seems like it might be useful for the deploy validations,
>> and also for improving the quality of error reporting -- would you
>> mind taking a look?
>
>
> Yeah we may end up using the heat stack dump tool for some validations since
> it puts all the Heat data in one place.
>
> However, this seems like a great thing to be included in the openstack
> overcloud deploy command and/or the mistral workflow.
>
> I.e. after each deployment, we could run list_nodes_status to verify that
> the overcloud nodes are indeed running and accessible (before tempest) and
> dump the o-c-c logs to a known directory.

Cool!

Also don't forget about sbaker's amazing new error reporting tool
"openstack stack failures list overcloud" -- might be useful...

--Hugh


-- 
  ,   ,| Hugh Brock, hbr...@redhat.com
  )-_"""_-(| Director of Engineering, OpenStack Management
 ./ o\ /o \.   | TripleO: Install, configure, and scale OpenStack.
. \__/ \__/ .  | http://rdoproject.org, http://tripleo.org
...   V   ...  |
... - - - ...  | "I know that you believe you understand what you
 .   - -   .   | think I said, but I'm not sure you realize that what
  `-.-´| you heard is not what I meant." --Robert McCloskey
 "TripleOwl"

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [E] [TripleO] scripts to do post deployment analysis of an overcloud

[Hugh Brock]

On Wed, Aug 3, 2016 at 4:49 PM, Joe Talerico <jtale...@redhat.com> wrote:
> On Wed, Jul 27, 2016 at 2:04 AM, Hugh Brock <hbr...@redhat.com> wrote:
>> On Jul 26, 2016 8:08 PM, "Gordon, Kent" <kent.gor...@verizonwireless.com>
>> wrote:
>>>
>>>
>>>
>>>
>>>
>>>
>>> > -Original Message-
>>> > From: Gonéri Le Bouder [mailto:gon...@lebouder.net]
>>> > Sent: Tuesday, July 26, 2016 12:24 PM
>>> > To: openstack-dev@lists.openstack.org
>>> > Subject: [E] [openstack-dev] [TripleO] scripts to do post deployment
>>> > analysis
>>> > of an overcloud
>>> >
>>> > Hi all,
>>> >
>>> > For the Distributed-CI[0] project, we did two scripts[1] that we use to
>>> > extract
>>>
>>> Links not included in message
>>>
>>> > information from an overcloud.
>>> > We use this information to improve the readability of the deployment
>>> > logs.
>>> > I attached an example to show how we use the extracted stack
>>> > information.
>>> >
>>> > Now my question, do you know some other tools that we can use to do this
>>> > kind of anaylsis?
>>> > --
>>> > Gonéri Le Bouder
>>>
>>> Kent S. Gordon
>>
>> Joe, any overlap with Browbeat here?
>>
>> -Hugh
>
> Hey Hugh- Not from what I can tell... Any reason this tool couldn't be
> built into the openstack-api?
>
> This seems to be looking at the heat information? Browbeat will create
> a ansible inventory and login to a Overcloud deployment and check on
> settings and the cluster vs looking at heat.
>
> Joe

Tomaš, this seems like it might be useful for the deploy validations,
and also for improving the quality of error reporting -- would you
mind taking a look?

--Hugh


-- 
  ,   ,| Hugh Brock, hbr...@redhat.com
  )-_"""_-(| Director of Engineering, OpenStack Management
 ./ o\ /o \.   | TripleO: Install, configure, and scale OpenStack.
. \__/ \__/ .  | http://rdoproject.org, http://tripleo.org
...   V   ...  |
... - - - ...  | "I know that you believe you understand what you
 .   - -   .   | think I said, but I'm not sure you realize that what
  `-.-´| you heard is not what I meant." --Robert McCloskey
 "TripleOwl"

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] service validation during deployment steps

[Hugh Brock]

On Jul 26, 2016 10:02 PM, "Emilien Macchi"  wrote:
>
> I would love to hear some feedback about $topic, thanks.

Ian, Rhys, Jacob - this sounds like a big step in the right direction to
me, what do you guys think?

-Hugh

>
> On Fri, Jul 15, 2016 at 11:31 AM, Emilien Macchi 
wrote:
> > Hi,
> >
> > Some people on the field brought interesting feedback:
> >
> > "As a TripleO User, I would like the deployment to stop immediately
> > after an resource creation failure during a step of the deployment and
> > be able to easily understand what service or resource failed to be
> > installed".
> >
> > Example:
> > If during step4 Puppet tries to deploy Neutron and OVS, but OVS fails
> > to start for some reasons, deployment should stop at the end of the
> > step.
> >
> > So there are 2 things in this user story:
> >
> > 1) Be able to run some service validation within a step deployment.
> > Note about the implementation: make the validation composable per
> > service (OVS, nova, etc) and not per role (compute, controller, etc).
> >
> > 2) Make this information readable and easy to access and understand
> > for our users.
> >
> > I have a proof-of-concept for 1) and partially 2), with the example of
> > OVS: https://review.openstack.org/#/c/342202/
> > This patch will make sure OVS is actually usable at step 4 by running
> > 'ovs-vsctl show' during the Puppet catalog and if it's working, it
> > will create a Puppet anchor. This anchor is currently not useful but
> > could be in future if we want to rely on it for orchestration.
> > I wrote the service validation in Puppet 2 years ago when doing Spinal
> > Stack with eNovance:
> >
https://github.com/openstack/puppet-openstacklib/blob/master/manifests/service_validation.pp
> > I think we could re-use it very easily, it has been proven to work.
> > Also, the code is within our Puppet profiles, so it's by design
> > composable and we don't need to make any connection with our current
> > services with some magic. Validation will reside within Puppet
> > manifests.
> > If you look my PoC, this code could even live in puppet-vswitch itself
> > (we already have this code for puppet-nova, and some others).
> >
> > Ok now, what if validation fails?
> > I'm testing it here: https://review.openstack.org/#/c/342205/
> > If you look at /var/log/messages, you'll see:
> >
> > Error:
/Stage[main]/Tripleo::Profile::Base::Neutron::Ovs/Openstacklib::Service_validation[openvswitch]/Exec[execute
> > openvswitch validation]/returns: change from notrun to 0 failed
> >
> > So it's pretty clear by looking at logs that openvswitch service
> > validation failed and something is wrong. You'll also notice in the
> > logs that deployed stopped at step 4 since OVS is not considered to
> > run.
> > It's partially addressing 2) because we need to make it more explicit
> > and readable. Dan Prince had the idea to use
> > https://github.com/ripienaar/puppet-reportprint to print a nice report
> > of Puppet catalog result (we haven't tried it yet). We could also use
> > Operational Tools later to monitor Puppet logs and find Service
> > validation failures.
> >
> >
> > So this email is a bootstrap of discussion, it's open for feedback.
> > Don't take my PoC as something we'll implement. It's an idea and I
> > think it's worth to look at it.
> > I like it for 2 reasons:
> > - the validation code reside within our profiles, so it's composable by
design.
> > - it's flexible and allow us to test everything. It can be a bash
> > script, a shell command, a Puppet resource (provider, service, etc).
> >
> > Thanks for reading so far,
> > --
> > Emilien Macchi
>
>
>
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [E] [TripleO] scripts to do post deployment analysis of an overcloud

[Hugh Brock]

On Jul 26, 2016 8:08 PM, "Gordon, Kent" 
wrote:
>
>
>
>
>
>
> > -Original Message-
> > From: Gonéri Le Bouder [mailto:gon...@lebouder.net]
> > Sent: Tuesday, July 26, 2016 12:24 PM
> > To: openstack-dev@lists.openstack.org
> > Subject: [E] [openstack-dev] [TripleO] scripts to do post deployment
analysis
> > of an overcloud
> >
> > Hi all,
> >
> > For the Distributed-CI[0] project, we did two scripts[1] that we use to
extract
>
> Links not included in message
>
> > information from an overcloud.
> > We use this information to improve the readability of the deployment
logs.
> > I attached an example to show how we use the extracted stack
information.
> >
> > Now my question, do you know some other tools that we can use to do this
> > kind of anaylsis?
> > --
> > Gonéri Le Bouder
>
> Kent S. Gordon

Joe, any overlap with Browbeat here?

-Hugh

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Should we have a TripleO API, or simply use Mistral?

[Hugh Brock]

work in the other projects there's a
> fair chance you are familiar with the Python clients.  How many people
> know Mistral YAML?
>
> Maybe I'm overestimating the Python knowledge in the community, and
> underestimating the Mistral knowledge, but I would bet we're talking
> order(s?) of magnitude in terms of the difference.  And I'm not saying
> learning Mistral is a huge task on its own, but it's one more thing in a
> project full of one more things.
>
>>
>> Looking at the prototypes Dan has done, it *is* possible to define a much
>> more strongly versioned API with Mistral than it is with Heat, which is the
>> main requirement in terms of lowering the bar (stable, relatively
>> opinionated by default and easy to use).
>>
>> Steve
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
* Hugh Brock, hbr...@redhat.com
* Senior Engineering Manager, Cloud Engineering
* RDO Manager: Install, configure, and scale OpenStack.
* http://rdoproject.org

"I know that you believe you understand what you think I said, but I'm
not sure you realize that what you heard is not what I meant."
--Robert McCloskey

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] HA CI job is green, please consider it when merging patches

[Hugh Brock]

Jirka, this is amazing work. Thanks to you, Giulio, Yanis, and everyone else 
who made it happen.

Now, on to network management... :) ...

Sent from my mobile, please pardon the top posting.

From: Jiří Stránský
Sent: May 22, 2015 1:50 AM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [TripleO] HA CI job is green, please consider it when 
merging patches

Hi all,

after an epic battle with bugs this week, we have a passing CI job for HA.

Looking at the jobs which ran during last night, the success rate is 
decent (14 all-green runs vs. just 1 run where HA job was the sole CI 
failure).

I'm a bit reluctant still to say let's make it voting right now, but i 
think we should be heading that way gradually. If you see a failed HA CI 
job from now on, there's some chance it points out some real issue. 
Please try to go through the logs before overriding its vote and merging 
a patch with red HA CI. If the patch in question is not critical with 
regards to time, recheck is an option :)


Thanks

Jirka

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] How to model resources in Heat

[Hugh Brock]

 On Jan 31, 2014, at 1:30 AM, Clint Byrum cl...@fewbar.com wrote:
 
 Excerpts from Zane Bitter's message of 2014-01-30 19:30:40 -0800:
 On 30/01/14 16:54, Clint Byrum wrote:
 I'm pretty sure it is useful to model images in Heat.
 
 Consider this scenario:
 
 
 resources:
   build_done_handle:
 type: AWS::CloudFormation::WaitConditionHandle
   build_done:
 type: AWS::CloudFormation::WaitCondition
 properties:
   handle: {Ref: build_done_handle}
   build_server:
 type: OS::Nova::Server
 properties:
   image: build-server-image
   userdata:
 join [ ,
   - #!/bin/bash\n
   - build_an_image\n
   - cfn-signal -s SUCCESS 
   - {Ref: build_done_handle}
   - \n]
   built_image:
 type: OS::Glance::Image
 depends_on: build_done
 properties:
   fetch_url: join [ , [http://;, {get_attribute: [ build_server, 
 fixed_ip ]}, /image_path]]
   actual_server:
 type: OS::Nova::Server
 properties:
   image: {Ref: built_image}
 
 
 Anyway, seems rather useful. Maybe I'm reaching.
 
 Well, consider that when this build is complete you'll still have the 
 server you used to build the image still sitting around. Of course you 
 can delete the stack to remove it - and along with it will go the image 
 in Glance. Still seem useful?
 
 No, not as such. However I have also discussed with other users having
 an OS::Heat::TemporaryServer which is deleted after a wait condition is
 signaled (resurrected on each update). This would be useful for hosting
 workflow code as the workflow doesn't actually need to be running all
 the time. It would also be useful for heat resources that want to run
 code that needs to be contained into their own VM/network such as the
 port probe thing that came up a few weeks ago.
 
 Good idea? I don't know. But it is the next logical step my brain keeps
 jumping to for things like this.
 
 (I'm conveniently ignoring the fact that you could have set 
 DeletionPolicy: Retain on the image to hack your way around this.)
 
 What you're looking for is a workflow service (I think it's called 
 Mistral this week?). A workflow service would be awesome, and Heat is 
 pretty awesome, but Heat is not a workflow service.
 
 Totally agree. I think workflow and orchestration have an unusual
 relationship though, because orchestration has its own workflow that
 users will sometimes need to defer to. This is why we use wait
 conditions, right?
 
 So yeah, Glance images in Heat might be kinda useful, but at best as a 
 temporary hack to fill in a gap because the Right Place to implement it 
 doesn't exist yet. That's why I feel ambivalent about it.
 
 I think you've nudged me away from optimistic at least closer to
 ambivalent as well.

We (RH tripleo folks) were having a similar conversation around Heat and stack 
upgrades the other day. There is unquestionably a workflow involving stack 
updates when a user goes to upgrade their overcloud, and it's awkward trying to 
shoehorn it into Heat (Steve Dake agreed). Our first thought was Tuskar should 
do that, but our second thought was Whatever the workflow service is should 
do that, and Tuskar should maybe provide a shorthand API for it.

I feel like we (tripleo) need to take a harder look at getting a working 
workflow thing available for our needs, soon...

--Hugh

 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev