date:20170710

On 07/10/2017 09:23 PM, James Slagle wrote:
> On Mon, Jul 10, 2017 at 2:54 PM, Giulio Fidente  wrote:
>> On 07/10/2017 07:06 PM, James Slagle wrote:
>>> On Mon, Jul 10, 2017 at 11:19 AM, Giulio Fidente  
>>> wrote:
 splitstack though requires changes in how the *existing* openstack
 services are deployed and we didn't want to do that just for the purpose
 of integrating ceph-ansible so I still believe (3) to be a sensible
 compromise to provide the needed functionalities and not breaking the
 existing deployment logic
>>>
>>> We might be talking about different definitions of "splitstack", as
>>> I'm not sure what changes are required for existing services. FWIW, I
>>> refer to what we do in CI with multinode to be splitstack in that the
>>> nodes are already provisioned and we deploy the services on those
>>> nodes using the same templates that we do for a "full" stack.
>>
>>> Those nodes could have just as easily been provisioned with our
>>> undercloud and the services deployed using 2 separate stacks, and that
>>> model works just as well.
>>
>> true, sorry for the misuse of the term splistack; the existing
>> splitstack implementation continues to work well and option (3), like
>> the others, can be plugged on top of it
>>
>> what I had in mind was instead the "split stack" scenario described by
>> Steven, where the orchestration steps are moved outside heat, this is
>> what we didn't have, still don't have and can be discussed at the PTG
> 
> Ok, thanks for clarifying. So when you're saying split-stack in this
> context, you imply just deploying a baremetal stack, then use whatever
> tool we want (or may develop) to deploy the service configuration.

yes but I am still assuming heat to be tool providing the per-role and
per-service settings, while not the tool orchestrating the steps anymore

I also don't think we should assume puppet or ansible to be "the
deployment tool"; the past seems to be telling us that we changed the
tool once already, later decided to use a new one fitting better our
needs for upgrades and yet resorted to a third more generic 'workflow
triggering' mechanism to decouple further some services configuration
from the general approach and I wouldn't give away flexibility easily
-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][TC] Application for inclusion of Glare in the list of official projects

2017-07-10 Thread Mikhail Fedosin

Thank you for asking this! It's really very important and interesting, so
I'm going to explain those things more detailed.

First, when we designed Glare, we kept in mind the compatibility with
Glance, and I can tell that Glance data from the database can be ported to
Glare with a simple script without any loss.

Second, APIs are very similar and map 1:1. The only one big difference is
that user has to perform activation manually after image file is uploaded.
I created a small table with the most popular API requests. You may notice
how similar both APIs are:
https://docs.google.com/document/d/18Tqad0NUPyFfHUo1KMr6bDDISpQtzacvZtEQIGhNkf4/edit?usp=sharing
Other changes are rather cosmetic. For instance, "queued" image status was
renamed to "drafted".

Third, all these changes can be hidden in Glare client. So if we try a
little, we can achieve 100% compatibility there, and other projects can use
Glare client instead of Glance's without even noticing the differences.

If projects use Glance without client, it means that some direct API
requests will need to be rewritten. But in any case, the number of
differences between Glance v1 and Glance v2 was much larger, and we
switched pretty smoothly. So I hope everything will be fine here, too.

Best,
Mike Fedosin

On Mon, Jul 10, 2017 at 9:55 PM, Joshua Harlow 
wrote:

> Ed Leafe wrote:
>
>> On Jul 10, 2017, at 5:06 AM, Mikhail Fedosin > > wrote:
>>
>> Given all the advantages and features of Glare, I believe that it can
>>> become the successful drop-in replacement.
>>>
>>
>> Can you clarify this? Let’s assume I have a decent-sized deployment
>> running Glance. If I were to remove Glance and replace it with Glare,
>> are you saying that nothing would break? Operators, users, scripts,
>> SDKs, etc., would all work unchanged?
>>
>
> Sounds interesting,
>
> Is there some kind of glance-compat API?
>
>
>> -- Ed Leafe
>>
>>
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][TC] Application for inclusion of Glare in the list of official projects

2017-07-10 Thread Monty Taylor


On 07/10/2017 01:21 PM, Ed Leafe wrote:
On Jul 10, 2017, at 5:06 AM, Mikhail Fedosin > wrote:


Given all the advantages and features of Glare, I believe that it can 
become the successful drop-in replacement.


Can you clarify this? Let’s assume I have a decent-sized deployment 
running Glance. If I were to remove Glance and replace it with Glare, 
are you saying that nothing would break? Operators, users, scripts, 
SDKs, etc., would all work unchanged?


I also have this question. The glance API is one of the most fundamental 
and basic APIs. You pretty much can't do anything useful on a cloud 
without touching it.


That said - it's not like glare couldn't also do those things - but I'd 
need to understand some real specifics about what a cloud switching from 
glance to glare looks like to the end user.


Also, we have a new upload API designed for glance that took a LARGE 
amount of wrangling to get consensus on. I'd also want to know what this 
situation looks like in glare, if image upload in glare supports all of 
the use-cases that we figured out image upload in glance needed to 
support. AND - there are folks who want import-from which was removed 
between glance v1 and v2. Does glare support something in this area?


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Fox, Kevin M

I think the migration path to something like kolla-kubernetes would be fine,
as you have total control over the orchestration piece, ansible and the config 
generation
ansible and since it is all containerized and TripleO production isn't, you 
should be able to
'upgrade' from non containtered to containered while leaving alone all the 
existing services as a 
roll back path. Something like read in the old config, tweak it a bit as 
needed, upload as configmaps. then helm install some kolla packages?

Thanks,
Kevin

From: Emilien Macchi [emil...@redhat.com]
Sent: Monday, July 10, 2017 12:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On Mon, Jul 10, 2017 at 6:19 AM, Steven Hardy  wrote:
[...]
> 1. How to perform end-to-end configuration via ansible (outside of
> heat, but probably still using data and possibly playbooks generated
> by heat)

I guess we're talking about removing Puppet from TripleO and use more
Ansible to manage configuration files.

This is somewhat related to what Flavio (and team) are currently investigating:
https://github.com/flaper87/tripleo-apb-roles/tree/master/keystone-apb

Also see this thread for more context:
http://lists.openstack.org/pipermail/openstack-dev/2017-June/118417.html

We could imagine these apb used by Split Stack 2 by applying the
software configuration (Ansible) to deploy OpenStack on already
deployed baremetal nodes.
One of the challenges here is how do we get the data from Heat to
generate Ansible vars.

[...]

>> I think if we can form some broad agreement before the PTG, we have a
>> chance at making some meaningful progress during Queens.
>
> Agreed, although we probably do need to make some more progress on
> some aspects of this for container minor updates that we'll need for
> Pike.

++ Thanks for bringing this James.

Some other thoughts:
* I also agree that TripleO Quickstart is a separated topic. I was
also confused why OOOQ was templating bash scripts and it has become
clear we needed a way to run exactly the commands in our documentation
without abstraction (please tell me if I'm wrong), therefore we had to
do these templates. We could have been a bit more granular (run cmds
in tasks instead of shell scripts) but I might have missed something
why we didn't do that way.

* Kayobe and Kolla are great tools, though TripleO is looking for a
path to migrate to Ansible in a backward compatible way. Throwing a
third grenade here - I think these tools are too opinionated to allow
us to simply use them. I think we should work toward re-using the
maximum of bits when it makes sense, but folks need to keep in mind we
need to support our existing production deployments, manage upgrades
etc. We're already using some bits from Kolla and our team is already
willing to collaborate with other deployments tools when it makes
sense.

* I agree with some comments in this thread when I read "TripleO would
be a tool to deploy OpenStack Infrastructure as split stacks", like
we're doing in our multinode jobs but even further. I'm interested by
the work done by Flavio and see how we could use Split Stack 2 to
deploy Kubernetes with Ansible (eventually without Mistral calling
Heat calling Mistral calling Ansible).

* It might sound like we want to add more complexity in TripleO but I
confirm James's goal which is a common goal in the team, is to reduce
the number of tools used by TripleO. In other words, we hope we can
e.g. remove Puppet to manage configuration files (which could be done
by Ansible), remove some workflows usually done by Heat but could be
done by Ansible as well, etc. The idea around forming plans to use
Ansible is excellent and we need to converge our efforts together so
we can address some of our operators's feedbacks.

Thanks,
--
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

2017-07-10 Thread Emilien Macchi

Sounds like we've got a bunch of positive feedback here - Thanks Alex
for your work and involvment, you're now tripleo-core in _all_ tripleo
projects.

On Mon, Jul 10, 2017 at 8:38 AM, Ben Nemec  wrote:
> +1 for sure
>
>
> On 07/07/2017 12:39 PM, Emilien Macchi wrote:
>>
>> Alex has demonstrated high technical and community skills in TripleO -
>> where he's already core on THT, instack-undercloud, and puppet-tripleo
>> - but also very involved in other repos.
>> I propose that we extend his core status to all TripleO projects and
>> of course trust him (like we trust all core members) to review patches
>> were we feel confortable with.
>>
>> He has shown an high interest in reviewed other TripleO projects and I
>> think he would be ready for this change.
>> As usual, this is an open proposal, any feedback is welcome.
>>
>> Thanks,
>>
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] July 14 CFP Deadline - Sydney Summit

2017-07-10 Thread Allison Price

Hi everyone,

Friendly reminder that the OpenStack Summit Sydney Call for Presentations 
closes THIS Friday, July 14 at 11:59 pm Pacific Time/ July 15 at 6:59 UTC. 
Don’t miss your chance to present in Sydney! View the new tracks here. 

SUBMIT HERE 


Cheers,
Allison


Allison Price
OpenStack Foundation
alli...@openstack.org


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Alex Schultz

On Fri, Jul 7, 2017 at 11:50 AM, James Slagle  wrote:
> I proposed a session for the PTG
> (https://etherpad.openstack.org/p/tripleo-ptg-queens) about forming a
> common plan and vision around Ansible in TripleO.
>
> I think it's important however that we kick this discussion off more
> broadly before the PTG, so that we can hopefully have some agreement
> for deeper discussions and prototyping when we actually meet in
> person.
>
> Right now, we have multiple uses of Ansible in TripleO:
>
> (0) tripleo-quickstart which follows the common and well accepted
> approach to bundling a set of Ansible playbooks/roles.
>
> (1) Mistral calling Ansible. This is the approach used by
> tripleo-validations where Mistral directly executes ansible playbooks
> using a dynamic inventory. The inventory is constructed from the
> server related stack outputs of the overcloud stack.
>
> (2) Ansible running playbooks against localhost triggered by the
> heat-config Ansible hook. This approach is used by
> tripleo-heat-templates for upgrade tasks and various tasks for
> deploying containers.
>
> (3) Mistral calling Heat calling Mistral calling Ansible. In this
> approach, we have Mistral resources in tripleo-heat-templates that are
> created as part of the overcloud stack and in turn, the created
> Mistral action executions run ansible. This has been prototyped with
> using ceph-ansible to install Ceph as part of the overcloud
> deployment, and some of the work has already landed. There are also
> proposed WIP patches using this approach to install Kubernetes.
>
> There are also some ideas forming around pulling the Ansible playbooks
> and vars out of Heat so that they can be rerun (or run initially)
> independently from the Heat SoftwareDeployment delivery mechanism:
>
> (4) https://review.openstack.org/#/c/454816/
>
> (5) Another idea I'd like to prototype is a local tool that runs on
> the undercloud and pulls all of the SoftwareDeployment data out of
> Heat as the stack is being created and generates corresponding Ansible
> playbooks to apply those deployments. Once a given playbook is
> generated by the tool, the tool would signal back to Heat that the
> deployment is complete. Heat then creates the whole stack without
> actually applying a single deployment to an overcloud node. At that
> point, Ansible (or Mistral->Ansible for an API) would be used to do
> the actual deployment of the Overcloud with the Undercloud as the
> ansible runner.
>
> All of this work has merit as we investigate longer term plans, and
> it's all at different stages with some being for dev/CI (0), some
> being used already in production (1 and 2), some just at the
> experimental stage (3 and 4), and some does not exist other than an
> idea (5).
>
> My intent with this mail is to start a discussion around what we've
> learned from these approaches and start discussing a consolidated plan
> around Ansible. And I'm not saying that whatever we come up with
> should only use Ansible a certain way. Just that we ought to look at
> how users/operators interact with Ansible and TripleO today and try
> and come up with the best solution(s) going forward.
>
> I think that (1) has been pretty successful, and my idea with (5)
> would use a similar approach once the playbooks were generated.
> Further, my idea with (5) would give us a fully backwards compatible
> solution with our existing template interfaces from
> tripleo-heat-templates. Longer term (or even in parallel for some
> time), the generated playbooks could stop being generated (and just
> exist in git), and we could consider moving away from Heat more
> permanently
>
> I recognize that saying "moving away from Heat" may be quite
> controversial. While it's not 100% the same discussion as what we are
> doing with Ansible, I think it is a big part of the discussion and if
> we want to continue with Heat as the primary orchestration tool in
> TripleO.
>
> I've been hearing a lot of feedback from various operators about how
> difficult the baremetal deployment is with Heat. While feedback about
> Ironic is generally positive, a lot of the negative feedback is around
> the Heat->Nova->Ironic interaction. And, if we also move more towards
> Ansible for the service deployment, I wonder if there is still a long
> term place for Heat at all.
>
> Personally, I'm pretty apprehensive about the approach taken in (3). I
> feel that it is a lot of complexity that could be done simpler if we
> took a step back and thought more about a longer term approach. I
> recognize that it's mostly an experiment/POC at this stage, and I'm
> not trying to directly knock down the approach. It's just that when I
> start to see more patches (Kubernetes installation) using the same
> approach, I figure it's worth discussing more broadly vs trying to
> have a discussion by -1'ing patch reviews, etc.
>
> I'm interested in all feedback of course. And I plan to take a shot at
> working on the prototype I

Re: [openstack-dev] [tc] Status update, July 7th

2017-07-10 Thread Emilien Macchi

On Fri, Jul 7, 2017 at 1:19 AM, Thierry Carrez  wrote:
[...}
> Who is up for discussing those items at our usual meeting slot
> time on Tuesday ?

Yes, count on me.

[...]

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On Mon, Jul 10, 2017 at 2:54 PM, Giulio Fidente  wrote:
> On 07/10/2017 07:06 PM, James Slagle wrote:
>> On Mon, Jul 10, 2017 at 11:19 AM, Giulio Fidente  wrote:
>>> splitstack though requires changes in how the *existing* openstack
>>> services are deployed and we didn't want to do that just for the purpose
>>> of integrating ceph-ansible so I still believe (3) to be a sensible
>>> compromise to provide the needed functionalities and not breaking the
>>> existing deployment logic
>>
>> We might be talking about different definitions of "splitstack", as
>> I'm not sure what changes are required for existing services. FWIW, I
>> refer to what we do in CI with multinode to be splitstack in that the
>> nodes are already provisioned and we deploy the services on those
>> nodes using the same templates that we do for a "full" stack.
>
>> Those nodes could have just as easily been provisioned with our
>> undercloud and the services deployed using 2 separate stacks, and that
>> model works just as well.
>
> true, sorry for the misuse of the term splistack; the existing
> splitstack implementation continues to work well and option (3), like
> the others, can be plugged on top of it
>
> what I had in mind was instead the "split stack" scenario described by
> Steven, where the orchestration steps are moved outside heat, this is
> what we didn't have, still don't have and can be discussed at the PTG

Ok, thanks for clarifying. So when you're saying split-stack in this
context, you imply just deploying a baremetal stack, then use whatever
tool we want (or may develop) to deploy the service configuration.


-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Emilien Macchi

On Mon, Jul 10, 2017 at 6:19 AM, Steven Hardy  wrote:
[...]
> 1. How to perform end-to-end configuration via ansible (outside of
> heat, but probably still using data and possibly playbooks generated
> by heat)

I guess we're talking about removing Puppet from TripleO and use more
Ansible to manage configuration files.

This is somewhat related to what Flavio (and team) are currently investigating:
https://github.com/flaper87/tripleo-apb-roles/tree/master/keystone-apb

Also see this thread for more context:
http://lists.openstack.org/pipermail/openstack-dev/2017-June/118417.html

We could imagine these apb used by Split Stack 2 by applying the
software configuration (Ansible) to deploy OpenStack on already
deployed baremetal nodes.
One of the challenges here is how do we get the data from Heat to
generate Ansible vars.

[...]

>> I think if we can form some broad agreement before the PTG, we have a
>> chance at making some meaningful progress during Queens.
>
> Agreed, although we probably do need to make some more progress on
> some aspects of this for container minor updates that we'll need for
> Pike.

++ Thanks for bringing this James.

Some other thoughts:
* I also agree that TripleO Quickstart is a separated topic. I was
also confused why OOOQ was templating bash scripts and it has become
clear we needed a way to run exactly the commands in our documentation
without abstraction (please tell me if I'm wrong), therefore we had to
do these templates. We could have been a bit more granular (run cmds
in tasks instead of shell scripts) but I might have missed something
why we didn't do that way.

* Kayobe and Kolla are great tools, though TripleO is looking for a
path to migrate to Ansible in a backward compatible way. Throwing a
third grenade here - I think these tools are too opinionated to allow
us to simply use them. I think we should work toward re-using the
maximum of bits when it makes sense, but folks need to keep in mind we
need to support our existing production deployments, manage upgrades
etc. We're already using some bits from Kolla and our team is already
willing to collaborate with other deployments tools when it makes
sense.

* I agree with some comments in this thread when I read "TripleO would
be a tool to deploy OpenStack Infrastructure as split stacks", like
we're doing in our multinode jobs but even further. I'm interested by
the work done by Flavio and see how we could use Split Stack 2 to
deploy Kubernetes with Ansible (eventually without Mistral calling
Heat calling Mistral calling Ansible).

* It might sound like we want to add more complexity in TripleO but I
confirm James's goal which is a common goal in the team, is to reduce
the number of tools used by TripleO. In other words, we hope we can
e.g. remove Puppet to manage configuration files (which could be done
by Ansible), remove some workflows usually done by Heat but could be
done by Ansible as well, etc. The idea around forming plans to use
Ansible is excellent and we need to converge our efforts together so
we can address some of our operators's feedbacks.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][TC] Application for inclusion of Glare in the list of official projects

2017-07-10 Thread Joshua Harlow


Ed Leafe wrote:

On Jul 10, 2017, at 5:06 AM, Mikhail Fedosin > wrote:


Given all the advantages and features of Glare, I believe that it can
become the successful drop-in replacement.


Can you clarify this? Let’s assume I have a decent-sized deployment
running Glance. If I were to remove Glance and replace it with Glare,
are you saying that nothing would break? Operators, users, scripts,
SDKs, etc., would all work unchanged?


Sounds interesting,

Is there some kind of glance-compat API?



-- Ed Leafe





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On 07/10/2017 07:06 PM, James Slagle wrote:
> On Mon, Jul 10, 2017 at 11:19 AM, Giulio Fidente  wrote:
>> On 07/10/2017 03:19 PM, Steven Hardy wrote:
>>> On Fri, Jul 7, 2017 at 6:50 PM, James Slagle  wrote:
>>
>> [...]
>>
>>> Yeah, I think the first step is to focus on a clean "split stack"
>>> model where the nodes/networks etc are still deployed via heat, then
>>> ansible handles the configuration of the nodes.
>>
>> +1
>>
>> as per my previous email, if splitstack was available we might have been
>> able to use that for the ceph-ansible integration : "if we had migrated
>> to splitstack already, it might have been possible"
> 
> Can you expand on what isn't available? I've primarily been the one
> working on different parts of splitstack, and I'm not sure what it
> can't do that you need it to do :).

the idea behind option (3) was to make it possible to run any mistral
workflow (or task) to deploy a service

we decoupled, on a per-service basis, how a given service is deployed
from the rest of the stack, yet maintained orchestration of the
overcloud deployment steps in heat; I know for sure that not everybody
liked this idea but it was the goal

as a result via option (3) you can deploy a new service in tripleo by
pointing it to a workflow ... and it doesn't matter if the workflow uses
ansible, puppet or simply returns 0

plus the workflow can be executed at a given deployment step, making it
possible to interleave its execution with the rest of the deployment
steps (the puppet apply steps); splitstack couldn't interleave the steps
and even if we made it to, we needed to add the parts to describe which
workflow/task needed to be run

but now that option (3) is implemented, assuming we move outside heat
the capability to collect and run tasks/workflows for a given service,
it'll be trivial to remove the "mistral > heat > mistral" loop, we'd
just need to execute the service workflows from the
$new_tool_driving_the_deployment_steps

>> splitstack though requires changes in how the *existing* openstack
>> services are deployed and we didn't want to do that just for the purpose
>> of integrating ceph-ansible so I still believe (3) to be a sensible
>> compromise to provide the needed functionalities and not breaking the
>> existing deployment logic
> 
> We might be talking about different definitions of "splitstack", as
> I'm not sure what changes are required for existing services. FWIW, I
> refer to what we do in CI with multinode to be splitstack in that the
> nodes are already provisioned and we deploy the services on those
> nodes using the same templates that we do for a "full" stack.

> Those nodes could have just as easily been provisioned with our
> undercloud and the services deployed using 2 separate stacks, and that
> model works just as well.

true, sorry for the misuse of the term splistack; the existing
splitstack implementation continues to work well and option (3), like
the others, can be plugged on top of it

what I had in mind was instead the "split stack" scenario described by
Steven, where the orchestration steps are moved outside heat, this is
what we didn't have, still don't have and can be discussed at the PTG

-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On Mon, Jul 10, 2017 at 11:37 AM, Lars Kellogg-Stedman  wrote:
> On Fri, Jul 7, 2017 at 1:50 PM, James Slagle  wrote:
>>
>> There are also some ideas forming around pulling the Ansible playbooks
>>
>> and vars out of Heat so that they can be rerun (or run initially)
>> independently from the Heat SoftwareDeployment delivery mechanism:
>
>
> I think the closer we can come to "the operator runs ansible-playbook to
> configure the overcloud" the better, but not because I think Ansible is
> inherently a great tool: rather, I think the many layers of indirection in
> our existing model make error reporting and diagnosis much more complicated
> that it needs to be.  Combined with Puppet's "fail as late as possible"
> model, this means that (a) operators waste time waiting for a deployment
> that is ultimately going to fail but hasn't yet, and (b) when it does fail,
> they need relatively intimate knowledge of our deployment tools to backtrack
> through logs and find the root cause of the failure.
>
> If we can offer a deployment mode that reduces the number of layers between
> the operator and the actions being performed on the hosts I think we would
> win on both fronts: faster failures and reporting errors as close as
> possible to the actual problem will result in less frustration across the
> board.
>
> I do like Steve's suggestion of a split model where Heat is responsible for
> instantiating OpenStack resources while Ansible is used to perform host
> configuration tasks.  Despite all the work done on Ansible's OpenStack
> modules, they feel inflexible and frustrating to work with when compared to
> Heat's state-aware, dependency ordered deployments.  A solution that allows
> Heat to output configuration that can subsequently be consumed by Ansible --
> either running manually or perhaps via Mistral for API-driven-deployments --
> seems like an excellent goal.  Using Heat as a "front-end" to the process
> means that we get to keep the parameter validation and documentation that is
> missing in Ansible, while still following the Unix philosophy of giving you
> enough rope to hang yourself if you really want it.

This is excellent input, thanks for providing it.

I think it lends itself towards suggesting that we may like to persue
(again) adding native Ironic resources to Heat. If those were written
in a way that also addressed some of the feedback about TripleO and
the baremetal deployment side, then we could continue to get the
advantages from Heat that you mention.

My personal opinion to date is that Ansible's os_ironic* modules are
superior in some ways to the Heat->Nova->Ironic model. However, just a
Heat->Ironic model may work in a way that has the advantages of both.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][TC] Application for inclusion of Glare in the list of official projects

2017-07-10 Thread Ed Leafe

On Jul 10, 2017, at 5:06 AM, Mikhail Fedosin  wrote:

> Given all the advantages and features of Glare, I believe that it can become 
> the successful drop-in replacement.

Can you clarify this? Let’s assume I have a decent-sized deployment running 
Glance. If I were to remove Glance and replace it with Glare, are you saying 
that nothing would break? Operators, users, scripts, SDKs, etc., would all work 
unchanged?

-- Ed Leafe

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

Excerpts from Anne Gentle's message of 2017-07-10 13:14:52 -0400:
> On Mon, Jul 10, 2017 at 1:11 PM, Doug Hellmann  wrote:
> > Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
> >> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> >>
> >> Of these, 'project' is static and should never change, so we don't need to
> >> attempt to guess them. Version and release are dynamic but I've noticed we
> >> don't actually use them anywhere in openstackdocstheme (the version you 
> >> see in
> >> the URL is actually an artefact of the build process and nothing to do with
> >> Sphinx). The closest thing we _do_ get to including version information is 
> >> the
> >> commit ID include in header of api-ref build pages [1], which is generated 
> >> by
> >> openstackdocstheme.
> >
> > It's surprising to me that we don't include the version string anywhere
> > on the page. Is that an oversight in the theme, or was it on purpose?
> 
> The theme shows "current" - however I noticed while reviewing this
> morning the nomenclature is "latest" in the URL, so I will make a
> change. Do you suggest "latest -n.n.n" as the string value, or simply
> "latest"?

I thought we would want to insert the version, as reported in the
Sphinx config, without any reference to current or latest or the
release series.

Perhaps we can replace the static text "Project home page" in the left
sidebar with "{{project}} {{version}}" (with or without "home page")
like in https://review.openstack.org/482230 ?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [docs] migration status

Progress


After more than 580 patches, the migration is moving along very
well. We do still have quite a few repositories that haven't been
updated [1], though, so please help us trim that list. It's really
important that we finish the migration before we create stable/pike
branches to avoid having to backport the content.

Recovering Old Content
--

We have started working on a tool [2] to recover "old" versions of
docs associated with tagged releases and stable branches by moving
it out of the /developer directory on the web server into the new
publishing structure. We will be coordinating with the infra team
over the next couple of weeks to move the content. I will send
another announcement when that is done.

The Theme Is Not Enough
---

Several project teams have landed the patch to move from oslosphinx
to openstackdocstheme. Please remember that that is only one of
several steps in the migration, and is not sufficient to complete
the work. The new templating system for docs.openstack.org relies
on a YAML file with booleans indicating when a project has specific
types of content (installation instructions, API reference and/or
guide, configuration reference, etc.). The flag allows the template
to insert a pre-defined URL matching the new documentation layout,
and a gate job verifies that the content exists before the link is
allowed. That means that if you have content published somewhere
other than where it is expected, it *will not be linked* from the
landing pages on docs.openstack.org.

In-Tree API Docs


We have had one or two projects who already have in-tree API
documentation that was not being published to developer.openstack.org
ask about having their links added to docs.openstack.org. That is
in process [3].

Theme Rendering Issues
--

A few folks have also raised issues with the appearance of the docs
after moving to openstackdocs theme. Good, you're reading! Please
files bugs [4] (or patches [5]) for these issues. The theme is still
being updated, and there will be lots of opportunities to address any
problems.

Doug

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-July/119483.html
[2] https://review.openstack.org/481710
[3] https://review.openstack.org/482188
[4] https://launchpad.net/openstack-manuals
[5] http://git.openstack.org/cgit/openstack/openstackdocstheme

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat][ironic][telemetry][dragonflow][freezer][kuryr][manila][mistral][monasca][neutron][ansible][congress][rally][senlin][storlets][zun][docs] repos without signs of migration startin

According to the dashboard, it looks like we still have almost 100
repositories with documentation that have no patches with the
doc-migration topic, indicating that they have not started moving
content or updating the theme. I have tried to tag those teams in the
subject, but I may have missed some. Please check the list below for a
repo owned by your team.

If you have completed the work and the dasbhoard script didn't pick
it up, please let me know so I can fix up the data.

Doug

openstack-dev/heat-cfnclient
openstack/bifrost
openstack/ceilometer-powervm
openstack/ceilometermiddleware
openstack/diskimage-builder
openstack/dragonflow
openstack/freezer-api
openstack/freezer-dr
openstack/freezer-web-ui
openstack/fuxi
openstack/fuxi-kubernetes
openstack/heat
openstack/heat-cfntools
openstack/heat-translator
openstack/instack
openstack/ironic-lib
openstack/karbor-dashboard
openstack/kolla-kubernetes
openstack/manila
openstack/manila-image-elements
openstack/manila-ui
openstack/mistral-dashboard
openstack/mistral-extra
openstack/mistral-lib
openstack/molteniron
openstack/monasca-statsd
openstack/monasca-transform
openstack/networking-hyperv
openstack/networking-midonet
openstack/networking-vsphere
openstack/neutron-lbaas
openstack/neutron-lbaas-dashboard
openstack/octavia-dashboard
openstack/openstack-ansible-apt_package_pinning
openstack/openstack-ansible-ceph_client
openstack/openstack-ansible-galera_client
openstack/openstack-ansible-galera_server
openstack/openstack-ansible-haproxy_server
openstack/openstack-ansible-lxc_container_create
openstack/openstack-ansible-lxc_hosts
openstack/openstack-ansible-memcached_server
openstack/openstack-ansible-openstack_hosts
openstack/openstack-ansible-openstack_openrc
openstack/openstack-ansible-os_aodh
openstack/openstack-ansible-os_barbican
openstack/openstack-ansible-os_ceilometer
openstack/openstack-ansible-os_cinder
openstack/openstack-ansible-os_designate
openstack/openstack-ansible-os_glance
openstack/openstack-ansible-os_gnocchi
openstack/openstack-ansible-os_heat
openstack/openstack-ansible-os_horizon
openstack/openstack-ansible-os_ironic
openstack/openstack-ansible-os_keystone
openstack/openstack-ansible-os_magnum
openstack/openstack-ansible-os_molteniron
openstack/openstack-ansible-os_neutron
openstack/openstack-ansible-os_nova
openstack/openstack-ansible-os_octavia
openstack/openstack-ansible-os_rally
openstack/openstack-ansible-os_sahara
openstack/openstack-ansible-os_swift
openstack/openstack-ansible-os_tempest
openstack/openstack-ansible-os_trove
openstack/openstack-ansible-pip_install
openstack/openstack-ansible-plugins
openstack/openstack-ansible-rabbitmq_server
openstack/openstack-ansible-repo_build
openstack/openstack-ansible-repo_server
openstack/openstack-ansible-rsyslog_client
openstack/openstack-ansible-rsyslog_server
openstack/openstack-ansible-security
openstack/os-net-config
openstack/os-win
openstack/osc-placement
openstack/oslosphinx
openstack/pycadf
openstack/python-congressclient
openstack/python-ironic-inspector-client
openstack/python-manilaclient
openstack/python-octaviaclient
openstack/python-saharaclient
openstack/python-tricircleclient
openstack/python-tripleoclient
openstack/python-vitrageclient
openstack/python-zaqarclient
openstack/python-zunclient
openstack/rally
openstack/searchlight-ui
openstack/senlin
openstack/storlets
openstack/sushy
openstack/sushy-tools
openstack/tosca-parser
openstack/virtualbmc
openstack/watcher-dashboard
openstack/yaql
openstack/zun

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Anne Gentle

On Mon, Jul 10, 2017 at 1:11 PM, Doug Hellmann  wrote:
> Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
>> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
>> > tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
>> > remove the feature at this point. However, this is going to necessitate 
>> > some
>> > mechanical changes for most projects with docs and this mail serves as a
>> > heads up and request for input before we proceed.
>>
>> [snip]
>>
>> > Here are the features that the plugin provides, along with the current
>> > migration plans:
>>
>> ...
>>
>> > - Automatically sets project name and version using pbr's machinery
>> >
>> >   Try to set this from 'openstackdocstheme'. If this isn't possible, folks
>> >   will need to add some some lines to 'conf.py' like keystoneauth does [7]
>>
>> I've been looking into this particular issue further, and the more I think
>> about it, the less necessary it seems. There are three configuration options
>> currently set by pbr:
>>
>> - project (the project name)
>> - version (the full version, including alpha/beta/rc tags)
>> - release (the short X.Y version)
>>
>> Of these, 'project' is static and should never change, so we don't need to
>> attempt to guess them. Version and release are dynamic but I've noticed we
>> don't actually use them anywhere in openstackdocstheme (the version you see 
>> in
>> the URL is actually an artefact of the build process and nothing to do with
>> Sphinx). The closest thing we _do_ get to including version information is 
>> the
>> commit ID include in header of api-ref build pages [1], which is generated by
>> openstackdocstheme.
>
> It's surprising to me that we don't include the version string anywhere
> on the page. Is that an oversight in the theme, or was it on purpose?

The theme shows "current" - however I noticed while reviewing this
morning the nomenclature is "latest" in the URL, so I will make a
change. Do you suggest "latest -n.n.n" as the string value, or simply
"latest"?

Anne

>
>> Given this fact, I think pbr is a providing a solution in search of problem
>> here, and this feature can in fact go quietly into the night with no further
>> ado.
>>
>> Thoughts?
>> Stephen
>>
>> PS: If we really did want to include a version in the docs in the future, I
>> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
>> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
>> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
>> these
>> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
>> think this is necessary.
>
> Relying on anything outside of the repo won't work for packagers who
> build from source outside of our CI system.
>
>> PPS: I have not responded to the other replies to this mail yet because Red
>> Hat's email servers have decided not to send me replies to my own posts on
>> openstack-dev. I have seen them though and will reply when I figure out how 
>> to
>> get mboxes.
>>
>> [1] https://developer.openstack.org/api-ref/compute/
>> [2] 
>> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
>> 54-L60
>> [3] 
>> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
>> ipts/run-docs.sh#L20
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 

Read my blog: justwrite.click
Subscribe to Docs|Code: docslikecode.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Mark Goddard

I'll throw a second grenade in.

Kayobe[1][2] is an OpenStack deployment tool based on kolla-ansible that
adds sounds in some ways similar to what you're describing. It roughly
follows the TripleO undercloud/overcloud model, with Bifrost used to deploy
the overcloud. Kayobe augments kolla-ansible with ansible playbooks for
configuration of the undercloud and overcloud hosts - networking, docker
storage, LVM. It also provides automation of some common workflows. There's
currently a focus on baremetal and scientific computing, but that's only
because that's been the focus up to now. Users drive kayobe using a CLI
which is mostly a wrapper around ansible-playbook.

I'm not suggesting that everyone should discard TripleO and adopt Kayobe -
clearly TripleO is a lot more mature. That said, if TripleO wants to move
to a more ansible-centric architecture, it might be prudent to see how
similar projects work, and if possible, share some code.

[1] https://github.com/stackhpc/kayobe
[2] http://kayobe.readthedocs.io/en/latest/

On 10 July 2017 at 17:44, Michał Jastrzębski  wrote:

> Hey,
>
> I'll just throw a grenade (pun intended) into your discussion - sorry!
> How about kolla-kubernetes? State awareness is done by kubernetes,
> it's designed for containers and we already have most of services
> ready and we'll be running ansible inside containers on top of k8s,
> for all the things that k8s is not natively good at. Sounds like
> somewhat you describe just switch heat with k8s.
>
> Cheers,
> Michal
>
> On 10 July 2017 at 08:37, Lars Kellogg-Stedman  wrote:
> > On Fri, Jul 7, 2017 at 1:50 PM, James Slagle 
> wrote:
> >>
> >> There are also some ideas forming around pulling the Ansible playbooks
> >>
> >> and vars out of Heat so that they can be rerun (or run initially)
> >> independently from the Heat SoftwareDeployment delivery mechanism:
> >
> >
> > I think the closer we can come to "the operator runs ansible-playbook to
> > configure the overcloud" the better, but not because I think Ansible is
> > inherently a great tool: rather, I think the many layers of indirection
> in
> > our existing model make error reporting and diagnosis much more
> complicated
> > that it needs to be.  Combined with Puppet's "fail as late as possible"
> > model, this means that (a) operators waste time waiting for a deployment
> > that is ultimately going to fail but hasn't yet, and (b) when it does
> fail,
> > they need relatively intimate knowledge of our deployment tools to
> backtrack
> > through logs and find the root cause of the failure.
> >
> > If we can offer a deployment mode that reduces the number of layers
> between
> > the operator and the actions being performed on the hosts I think we
> would
> > win on both fronts: faster failures and reporting errors as close as
> > possible to the actual problem will result in less frustration across the
> > board.
> >
> > I do like Steve's suggestion of a split model where Heat is responsible
> for
> > instantiating OpenStack resources while Ansible is used to perform host
> > configuration tasks.  Despite all the work done on Ansible's OpenStack
> > modules, they feel inflexible and frustrating to work with when compared
> to
> > Heat's state-aware, dependency ordered deployments.  A solution that
> allows
> > Heat to output configuration that can subsequently be consumed by
> Ansible --
> > either running manually or perhaps via Mistral for
> API-driven-deployments --
> > seems like an excellent goal.  Using Heat as a "front-end" to the process
> > means that we get to keep the parameter validation and documentation
> that is
> > missing in Ansible, while still following the Unix philosophy of giving
> you
> > enough rope to hang yourself if you really want it.
> >
> > --
> > Lars Kellogg-Stedman 
> >
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.op
> enstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] notification update week 28

2017-07-10 Thread Matt Riedemann


On 7/10/2017 3:22 AM, Balazs Gibizer wrote:

Hi,

Here is the status update / focus setting mail about notification work
for week 28.

Bugs

[Undecided] https://bugs.launchpad.net/nova/+bug/1684860 Versioned
server notifications don't include updated_at
Takashi's fix needs a second +2 https://review.openstack.org/#/c/475276/

[Low] https://bugs.launchpad.net/nova/+bug/1696152 nova notifications
use nova-api as binary name instead of nova-osapi_compute
Agreed not to change the binary name in the notifications. Instead we
make an enum for that name to show that the name is intentional.
Patch has been proposed:  https://review.openstack.org/#/c/476538/

[Undecided] https://bugs.launchpad.net/nova/+bug/1702667 publisher_id of 
the versioned instance.update notification is not consistent with other 
notifications
The inconsistency of publisher_ids was revealed by #1696152. Fix has 
been proposed https://review.openstack.org/#/c/480984


[Undecided] https://bugs.launchpad.net/nova/+bug/1699115 api.fault
notification is never emitted
Still no response on the ML thread about the way forward.
http://lists.openstack.org/pipermail/openstack-dev/2017-June/118639.html

[Undecide] https://bugs.launchpad.net/nova/+bug/1700496 Notifications
are emitted per-cell instead of globally
Fix is to configure a global MQ endpoint for the notifications in cells
v2. Patch is being worked on: https://review.openstack.org/#/c/477556/

Versioned notification transformation
-
There is quite a long list of ready notification transformations for the 
cores to look at:
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/versioned-notification-transformation-pike+label:Code-Review%253E%253D%252B1+label:Verified%253E%253D1+AND+NOT+label:Code-Review%253C0 



If you are affraid of the long list then here is a short list of live 
migration related transformations:

* https://review.openstack.org/#/c/480214/
* https://review.openstack.org/#/c/420453/
* https://review.openstack.org/#/c/480119/
* https://review.openstack.org/#/c/469784/

Searchlight integration
---
bp additional-notification-fields-for-searchlight
~
The BDM addition needs core review, it just lost +2 due to a rebase:
https://review.openstack.org/#/c/448779/

Besides the BDM patch we are still missing the Add tags to
instance.create Notification https://review.openstack.org/#/c/459493/
patch but that depends on supporting tags and instance boot
https://review.openstack.org/#/c/394321/ which is still not ready.


One of my goals for this week is to get these two done so we can close 
out both of those blueprints.




Small improvements
~~
* https://review.openstack.org/#/c/428199/ Improve assertJsonEqual
error reporting
* https://review.openstack.org/#/q/topic:refactor-notification-samples
Factor out duplicated notification sample data
This is a start of a longer patch series to deduplicate notification
sample data. The third patch already shows how much sample data can be
deleted from nova tree. We added a minimal hand rolled json ref
implementation to notification sample test as the existing python json
ref implementations are not well maintained.

Weekly meeting
--
The notification subteam holds it's weekly meeting on Tuesday 17:00 UTC
on openstack-meeting-4. The next meeting will be held on 11th of July.
https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170711T17

Cheers,
gibi




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> > tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
> > remove the feature at this point. However, this is going to necessitate some
> > mechanical changes for most projects with docs and this mail serves as a
> > heads up and request for input before we proceed.
> 
> [snip]
> 
> > Here are the features that the plugin provides, along with the current
> > migration plans:
> 
> ...
> 
> > - Automatically sets project name and version using pbr's machinery
> > 
> >   Try to set this from 'openstackdocstheme'. If this isn't possible, folks 
> >   will need to add some some lines to 'conf.py' like keystoneauth does [7]
> 
> I've been looking into this particular issue further, and the more I think
> about it, the less necessary it seems. There are three configuration options
> currently set by pbr:
> 
> - project (the project name)
> - version (the full version, including alpha/beta/rc tags)
> - release (the short X.Y version)
> 
> Of these, 'project' is static and should never change, so we don't need to
> attempt to guess them. Version and release are dynamic but I've noticed we
> don't actually use them anywhere in openstackdocstheme (the version you see in
> the URL is actually an artefact of the build process and nothing to do with
> Sphinx). The closest thing we _do_ get to including version information is the
> commit ID include in header of api-ref build pages [1], which is generated by
> openstackdocstheme.

It's surprising to me that we don't include the version string anywhere
on the page. Is that an oversight in the theme, or was it on purpose?

> Given this fact, I think pbr is a providing a solution in search of problem
> here, and this feature can in fact go quietly into the night with no further
> ado.
> 
> Thoughts?
> Stephen
> 
> PS: If we really did want to include a version in the docs in the future, I
> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
> these
> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
> think this is necessary.

Relying on anything outside of the repo won't work for packagers who
build from source outside of our CI system.

> PPS: I have not responded to the other replies to this mail yet because Red
> Hat's email servers have decided not to send me replies to my own posts on
> openstack-dev. I have seen them though and will reply when I figure out how to
> get mboxes.
> 
> [1] https://developer.openstack.org/api-ref/compute/
> [2] 
> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
> 54-L60
> [3] 
> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
> ipts/run-docs.sh#L20
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][TC] Application for inclusion of Glare in the list of official projects

2017-07-10 Thread Kendall Nelson

Hello Mike :)

WRT the PTG, if you get approved as an official project before the end of
the month, we would love to have you!

The PTG isn't formatted the same as the Forum though. The PTG is more
project focused technical discussions amongst project contributors- think
midcycle rather than design summit or forum presentations. That being said,
we have the space for Glare contributors to meet for Wed- Thurs or
Wed-Friday if you think you can get enough people together to make it worth
while :)

Good luck with the last steps to becoming an official project!

-Kendall Nelson (diablo_rojo)

On Mon, Jul 10, 2017 at 3:10 AM Mikhail Fedosin  wrote:

> Hello!
>
> Recently I applied for inclusion of Glare in the list of official
> OpenStack projects: https://review.openstack.org/#/c/479285/ In this
> regard, I would like to ask the community what other things can be improved
> in the project to meet all the necessary requirements.
>
> Also in this thread I would like to discuss the role of the project in
> OpenStack. Lately opinions have been increasingly expressed that Glance no
> longer meets the current needs of OpenStack and should be replaced in the
> foreseeable future.  And as you know, initially Glare was designed as a new
> improved version of Glance, which works with all types of data, not just
> with virtual machine images. Given all the advantages and features of
> Glare, I believe that it can become the successful drop-in replacement.
> Glare already has parity in terms of features with Glance and can be used
> by Nova as a means for storing images. In addition to this we are
> developing a number of other interesting things that will help many
> projects to store their data in a convenient and reliable way. Therefore, I
> propose to transfer Glance to the status of stable and supported project
> starting from the next cycle, and to declare Glare as a new version in the
> medium term.
>
> In the end I would like to ask about PTG - I would be happy to present
> Glare at one or two sessions in Denver, if I get this opportunity.
>
> Thank you in advance for your comments!
>
> Best,
> Mike Fedosin
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On Mon, Jul 10, 2017 at 11:19 AM, Giulio Fidente  wrote:
> On 07/10/2017 03:19 PM, Steven Hardy wrote:
>> On Fri, Jul 7, 2017 at 6:50 PM, James Slagle  wrote:
>
> [...]
>
>> Yeah, I think the first step is to focus on a clean "split stack"
>> model where the nodes/networks etc are still deployed via heat, then
>> ansible handles the configuration of the nodes.
>
> +1
>
> as per my previous email, if splitstack was available we might have been
> able to use that for the ceph-ansible integration : "if we had migrated
> to splitstack already, it might have been possible"

Can you expand on what isn't available? I've primarily been the one
working on different parts of splitstack, and I'm not sure what it
can't do that you need it to do :).

>
> splitstack though requires changes in how the *existing* openstack
> services are deployed and we didn't want to do that just for the purpose
> of integrating ceph-ansible so I still believe (3) to be a sensible
> compromise to provide the needed functionalities and not breaking the
> existing deployment logic

We might be talking about different definitions of "splitstack", as
I'm not sure what changes are required for existing services. FWIW, I
refer to what we do in CI with multinode to be splitstack in that the
nodes are already provisioned and we deploy the services on those
nodes using the same templates that we do for a "full" stack.

Those nodes could have just as easily been provisioned with our
undercloud and the services deployed using 2 separate stacks, and that
model works just as well.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] weekly meetings on #tripleo

2017-07-10 Thread Michele Baldessari

On Mon, Jul 10, 2017 at 11:36:03AM -0230, Brent Eagles wrote:
> +1 for giving it a try.

Agreed.

> 
> On Wed, Jul 5, 2017 at 2:26 PM, Emilien Macchi  wrote:
> 
> > After reading http://lists.openstack.org/pipermail/openstack-dev/2017-
> > June/118899.html
> > - we might want to collect TripleO's community feedback on doing
> > weekly meetings on #tripleo instead of #openstack-meeting-alt.
> >
> > I see some direct benefits:
> > - if you come up late in meetings, you could easily read backlog in
> > #tripleo
> > - newcomers not aware about the meeting channel wouldn't have to search
> > for it
> > - meeting would maybe get more activity and we would expose the
> > information more broadly
> >
> > Any feedback on this proposal is welcome before we make any change (or
> > not).
> >
> > Thanks,
> > --
> > Emilien Macchi
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Michele Baldessari
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On Mon, Jul 10, 2017 at 9:19 AM, Steven Hardy  wrote:
> On Fri, Jul 7, 2017 at 6:50 PM, James Slagle  wrote:
> Yeah so my idea with (4), and subsequent patches such as[1] is to
> gradually move the deploy steps performed to configure services (on
> baremetal and in containers) to a single ansible playbook.
>
> There's currently still heat orchestration around the host preparation
> (although this is performed via ansible) and iteration over each step
> (where we re-apply the same deploy-steps playbook with an incrementing
> step variable, but this could be replaced by e.g an ansible or mistral
> loop), but my idea was to enable end-to-end configuration of nodes via
> ansible-playbook, without the need for any special tooks (e.g we
> refactor t-h-t enough that we don't need any special tools, and we
> make deploy-steps-playbook.yaml the only method of deployment (for
> baremetal and container cases)
>
> [1] https://review.openstack.org/#/c/462211/
>
>> All of this work has merit as we investigate longer term plans, and
>> it's all at different stages with some being for dev/CI (0), some
>> being used already in production (1 and 2), some just at the
>> experimental stage (3 and 4), and some does not exist other than an
>> idea (5).
>
> I'd like to get the remaining work for (4) done so it's a supportable
> option for minor updates, but there's still a bit more t-h-t
> refactoring required to enable it I think, but I think we're already
> pretty close to being able to run end-to-end ansible for most of the
> PostDeploy steps without any special tooling.

Thanks for this context, I think it helps clarify where we could be
going with these patches. I'll take a closer look at what you've done
so far.

I think I will missing the point of whether the playbooks would still
run localhost mode on each node, or if the idea would be that we could
eventually work towards a central ansible "runner" (such as the
undercloud) that could execute all the playbooks.

It sounds the latter is possibly just an iterative step beyond the
former, so I think like where this approach is going.

>> My intent with this mail is to start a discussion around what we've
>> learned from these approaches and start discussing a consolidated plan
>> around Ansible. And I'm not saying that whatever we come up with
>> should only use Ansible a certain way. Just that we ought to look at
>> how users/operators interact with Ansible and TripleO today and try
>> and come up with the best solution(s) going forward.
>>
>> I think that (1) has been pretty successful, and my idea with (5)
>> would use a similar approach once the playbooks were generated.
>> Further, my idea with (5) would give us a fully backwards compatible
>> solution with our existing template interfaces from
>> tripleo-heat-templates. Longer term (or even in parallel for some
>> time), the generated playbooks could stop being generated (and just
>> exist in git), and we could consider moving away from Heat more
>> permanently
>
> Yeah I think working towards aligning more TripleO configuration with
> the approach taken by tripleo-validations is fine, and we can e.g add
> more heat generated data about the nodes to the dynamic ansible
> inventory:
>
> https://github.com/openstack/tripleo-validations/blob/master/tripleo_validations/inventory.py
>
> We've been gradually adding data there, which I hope will enable a
> cleaner "split stack", where the nodes are deployed via heat, then
> ansible can do the configuration based on data exposed via stack
> outputs (which again is a pattern that I think has been proven to work
> quite well for tripleo-validations, and is also something I've been
> using locally for dev testing quite successfully).
>
>> I recognize that saying "moving away from Heat" may be quite
>> controversial. While it's not 100% the same discussion as what we are
>> doing with Ansible, I think it is a big part of the discussion and if
>> we want to continue with Heat as the primary orchestration tool in
>> TripleO.
>
> Yeah, I think the first step is to focus on a clean "split stack"
> model where the nodes/networks etc are still deployed via heat, then
> ansible handles the configuration of the nodes.
>
> In the long term I could see benefits in a "tripleo lite" model,
> where, say, we only used mistral+Ironic+ansible, but IMO we're not at
> the point yet where that's achievable, primarily because there's
> coupling between the heat parameter interfaces and multiple
> integrations we can't break (e.g users with environment files,
> tripleo-ui, vendor integrations, etc).
>
> It's a good discussion to kick off regardless though, so personally
> I'd like to focus on these as the first "baby steps":
>
> 1. How to perform end-to-end configuration via ansible (outside of
> heat, but probably still using data and possibly playbooks generated
> by heat)
>
> 2. How to deploy nodes directly via Ironic, with a mistral workflow
> (e.g no

Re: [openstack-dev] [kolla][kolla-ansible] Proposing Surya (spsurya) for core

2017-07-10 Thread Mauricio Lima

+1

2017-06-28 13:07 GMT-03:00 Swapnil Kulkarni :

> On Wed, Jun 14, 2017 at 9:16 PM, Michał Jastrzębski 
> wrote:
> > Hello,
> >
> > With great pleasure I'm kicking off another core voting to
> > kolla-ansible and kolla teams:) this one is about spsurya. Voting will
> > be open for 2 weeks (till 28th Jun).
> >
> > Consider this mail my +1 vote, you know the drill:)
> >
> > Regards,
> > Michal
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> +1
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Anne Gentle

On Mon, Jul 10, 2017 at 12:20 PM,   wrote:
> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
>> tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
>> remove the feature at this point. However, this is going to necessitate some
>> mechanical changes for most projects with docs and this mail serves as a
>> heads up and request for input before we proceed.
>
> [snip]
>
>> Here are the features that the plugin provides, along with the current
>> migration plans:
>
> ...
>
>> - Automatically sets project name and version using pbr's machinery
>>
>>   Try to set this from 'openstackdocstheme'. If this isn't possible, folks
>>   will need to add some some lines to 'conf.py' like keystoneauth does [7]
>
> I've been looking into this particular issue further, and the more I think
> about it, the less necessary it seems. There are three configuration options
> currently set by pbr:
>
> - project (the project name)
> - version (the full version, including alpha/beta/rc tags)
> - release (the short X.Y version)
>
> Of these, 'project' is static and should never change, so we don't need to
> attempt to guess them. Version and release are dynamic but I've noticed we
> don't actually use them anywhere in openstackdocstheme (the version you see in
> the URL is actually an artefact of the build process and nothing to do with
> Sphinx). The closest thing we _do_ get to including version information is the
> commit ID include in header of api-ref build pages [1], which is generated by
> openstackdocstheme.

The 1.12.0 release of openstackdocstheme provides an option to get to
more than the current version of a project's docs. It acquires the
values by querying the most recent five git tags:

https://review.openstack.org/#/c/477639/

Anne

>
> Given this fact, I think pbr is a providing a solution in search of problem
> here, and this feature can in fact go quietly into the night with no further
> ado.
>
> Thoughts?
> Stephen
>
> PS: If we really did want to include a version in the docs in the future, I
> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
> these
> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
> think this is necessary.
>
> PPS: I have not responded to the other replies to this mail yet because Red
> Hat's email servers have decided not to send me replies to my own posts on
> openstack-dev. I have seen them though and will reply when I figure out how to
> get mboxes.
>
> [1] https://developer.openstack.org/api-ref/compute/
> [2] 
> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
> 54-L60
> [3] 
> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
> ipts/run-docs.sh#L20
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 

Read my blog: justwrite.click
Subscribe to Docs|Code: docslikecode.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Michał Jastrzębski

Hey,

I'll just throw a grenade (pun intended) into your discussion - sorry!
How about kolla-kubernetes? State awareness is done by kubernetes,
it's designed for containers and we already have most of services
ready and we'll be running ansible inside containers on top of k8s,
for all the things that k8s is not natively good at. Sounds like
somewhat you describe just switch heat with k8s.

Cheers,
Michal

On 10 July 2017 at 08:37, Lars Kellogg-Stedman  wrote:
> On Fri, Jul 7, 2017 at 1:50 PM, James Slagle  wrote:
>>
>> There are also some ideas forming around pulling the Ansible playbooks
>>
>> and vars out of Heat so that they can be rerun (or run initially)
>> independently from the Heat SoftwareDeployment delivery mechanism:
>
>
> I think the closer we can come to "the operator runs ansible-playbook to
> configure the overcloud" the better, but not because I think Ansible is
> inherently a great tool: rather, I think the many layers of indirection in
> our existing model make error reporting and diagnosis much more complicated
> that it needs to be.  Combined with Puppet's "fail as late as possible"
> model, this means that (a) operators waste time waiting for a deployment
> that is ultimately going to fail but hasn't yet, and (b) when it does fail,
> they need relatively intimate knowledge of our deployment tools to backtrack
> through logs and find the root cause of the failure.
>
> If we can offer a deployment mode that reduces the number of layers between
> the operator and the actions being performed on the hosts I think we would
> win on both fronts: faster failures and reporting errors as close as
> possible to the actual problem will result in less frustration across the
> board.
>
> I do like Steve's suggestion of a split model where Heat is responsible for
> instantiating OpenStack resources while Ansible is used to perform host
> configuration tasks.  Despite all the work done on Ansible's OpenStack
> modules, they feel inflexible and frustrating to work with when compared to
> Heat's state-aware, dependency ordered deployments.  A solution that allows
> Heat to output configuration that can subsequently be consumed by Ansible --
> either running manually or perhaps via Mistral for API-driven-deployments --
> seems like an excellent goal.  Using Heat as a "front-end" to the process
> means that we get to keep the parameter validation and documentation that is
> missing in Ansible, while still following the Unix philosophy of giving you
> enough rope to hang yourself if you really want it.
>
> --
> Lars Kellogg-Stedman 
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread sfinucan

On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
> remove the feature at this point. However, this is going to necessitate some
> mechanical changes for most projects with docs and this mail serves as a
> heads up and request for input before we proceed.

[snip]

> Here are the features that the plugin provides, along with the current
> migration plans:

...

> - Automatically sets project name and version using pbr's machinery
> 
>   Try to set this from 'openstackdocstheme'. If this isn't possible, folks 
>   will need to add some some lines to 'conf.py' like keystoneauth does [7]

I've been looking into this particular issue further, and the more I think
about it, the less necessary it seems. There are three configuration options
currently set by pbr:

- project (the project name)
- version (the full version, including alpha/beta/rc tags)
- release (the short X.Y version)

Of these, 'project' is static and should never change, so we don't need to
attempt to guess them. Version and release are dynamic but I've noticed we
don't actually use them anywhere in openstackdocstheme (the version you see in
the URL is actually an artefact of the build process and nothing to do with
Sphinx). The closest thing we _do_ get to including version information is the
commit ID include in header of api-ref build pages [1], which is generated by
openstackdocstheme.

Given this fact, I think pbr is a providing a solution in search of problem
here, and this feature can in fact go quietly into the night with no further
ado.

Thoughts?
Stephen

PS: If we really did want to include a version in the docs in the future, I
think we could use information provided by ZUUL. I'm no ZUUL expert, but it
seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass these
via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
think this is necessary.

PPS: I have not responded to the other replies to this mail yet because Red
Hat's email servers have decided not to send me replies to my own posts on
openstack-dev. I have seen them though and will reply when I figure out how to
get mboxes.

[1] https://developer.openstack.org/api-ref/compute/
[2] https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
54-L60
[3] https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
ipts/run-docs.sh#L20

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

2017-07-10 Thread Ben Nemec


+1 for sure

On 07/07/2017 12:39 PM, Emilien Macchi wrote:

Alex has demonstrated high technical and community skills in TripleO -
where he's already core on THT, instack-undercloud, and puppet-tripleo
- but also very involved in other repos.
I propose that we extend his core status to all TripleO projects and
of course trust him (like we trust all core members) to review patches
were we feel confortable with.

He has shown an high interest in reviewed other TripleO projects and I
think he would be ready for this change.
As usual, this is an open proposal, any feedback is welcome.

Thanks,



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Lars Kellogg-Stedman

On Fri, Jul 7, 2017 at 1:50 PM, James Slagle  wrote:

> There are also some ideas forming around pulling the Ansible playbooks
>
and vars out of Heat so that they can be rerun (or run initially)
> independently from the Heat SoftwareDeployment delivery mechanism:
>

I think the closer we can come to "the operator runs ansible-playbook to
configure the overcloud" the better, but not because I think Ansible is
inherently a great tool: rather, I think the many layers of indirection in
our existing model make error reporting and diagnosis much more complicated
that it needs to be.  Combined with Puppet's "fail as late as possible"
model, this means that (a) operators waste time waiting for a deployment
that is ultimately going to fail but hasn't yet, and (b) when it does fail,
they need relatively intimate knowledge of our deployment tools to
backtrack through logs and find the root cause of the failure.

If we can offer a deployment mode that reduces the number of layers between
the operator and the actions being performed on the hosts I think we would
win on both fronts: faster failures and reporting errors as close as
possible to the actual problem will result in less frustration across the
board.

I do like Steve's suggestion of a split model where Heat is responsible for
instantiating OpenStack resources while Ansible is used to perform host
configuration tasks.  Despite all the work done on Ansible's OpenStack
modules, they feel inflexible and frustrating to work with when compared to
Heat's state-aware, dependency ordered deployments.  A solution that allows
Heat to output configuration that can subsequently be consumed by Ansible
-- either running manually or perhaps via Mistral for
API-driven-deployments -- seems like an excellent goal.  Using Heat as a
"front-end" to the process means that we get to keep the parameter
validation and documentation that is missing in Ansible, while still
following the Unix philosophy of giving you enough rope to hang yourself if
you really want it.

-- 
Lars Kellogg-Stedman 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

On 07/10/2017 03:19 PM, Steven Hardy wrote:
> On Fri, Jul 7, 2017 at 6:50 PM, James Slagle  wrote:

[...]

> Yeah, I think the first step is to focus on a clean "split stack"
> model where the nodes/networks etc are still deployed via heat, then
> ansible handles the configuration of the nodes.

+1

as per my previous email, if splitstack was available we might have been
able to use that for the ceph-ansible integration : "if we had migrated
to splitstack already, it might have been possible"

splitstack though requires changes in how the *existing* openstack
services are deployed and we didn't want to do that just for the purpose
of integrating ceph-ansible so I still believe (3) to be a sensible
compromise to provide the needed functionalities and not breaking the
existing deployment logic

note that I know of at least another case (the swift rings building)
which would benefit from being able to trigger a workflow during the
overcloud deployment and does not need to run ansible

[...]

>> Personally, I'm pretty apprehensive about the approach taken in (3). I
>> feel that it is a lot of complexity that could be done simpler if we
>> took a step back and thought more about a longer term approach. I
>> recognize that it's mostly an experiment/POC at this stage, and I'm
>> not trying to directly knock down the approach. It's just that when I
>> start to see more patches (Kubernetes installation) using the same
>> approach, I figure it's worth discussing more broadly vs trying to
>> have a discussion by -1'ing patch reviews, etc.
> 
> I agree, I think the approach in (3) is a stopgap until we can define
> a cleaner approach with less layers.

> IMO the first step towards that is likely to be a "split stack" which
> outputs heat data, then deployment configuration is performed via
> mistral->ansible just like we already do in (1).

given option (3) allows triggering of workflows during a particular
deployment step, it seems that option (1) would need to be revisited to
implement some sort of a loop in mistral, instead of heat, to provide
that same functionality ... which in the end moves the existing logic
from heat into mistral

>> I'm interested in all feedback of course. And I plan to take a shot at
>> working on the prototype I mentioned in (5) if anyone would like to
>> collaborate around that.
> 
> I'm very happy to collaborate, and this is quite closely related to
> the investigations I've been doing around enabling minor updates for
> containers.
> 
> Lets sync up about it, but as I mentioned above I'm not yet fully sold
> on a new translation tool, vs just more t-h-t refactoring to enable
> output of data directly consumable via ansible-playbook (which can
> then be run via operators, or heat, or mistral, or whatever).
I'd be happy to revisit the requirements around the ceph-ansible
integration as well, to see how those can still be met
-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread David Moreau Simard

That sounds like a good fit for an Ansible plugin to control how
variables or host inventories are designed [1] and is the intended use
case for extending Ansible behavior.

[1]: 
http://docs.ansible.com/ansible/dev_guide/developing_plugins.html#vars-plugins

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Mon, Jul 10, 2017 at 9:31 AM, Steven Hardy  wrote:
> On Sun, Jul 9, 2017 at 8:44 AM, Yolanda Robla Mota  
> wrote:
>> What i'd like to dig more is how Ansible and Heat can live together. And
>> what features do Heat offer that are not covered by Ansible as well? Is
>> there still the need to have Heat as the main engine, or could that be
>> replaced by Ansible totally in the future?
>
> The main interface provided by Heat which AFAIK cannot currently be
> replaced by Ansible is the parameters schema, where the template
> parameters are exposed (that include description, type and constraint
> data) in a format that is useful to e.g building the interfaces for
> tripleo-ui
>
> Ansible has a different approach to role/playbook parameters AFAICT,
> which is more a global namespace with no type validation, no way to
> include description data or tags with variable declarations, and no
> way to specify constraints (other than perhaps hainvg custom modules
> or playbook patterns that perform the validations early in the
> deployment).
>
> This is kind of similar to how the global namespace for hiera works
> with our puppet model, although that at least has the advantage of
> namespacing foo::something::variable, which again doesn't have a
> direct equivalent in the ansible role model AFAIK (happy to be
> corrected here, I'm not an ansible expert :)
>
> For these reasons (as mentioned in my reply to James), I think a first
> step of a "split stack" model where heat deploys the nodes/networks
> etc, then outputs data that can be consumed by Ansible is reasonable -
> it leaves the operator interfaces alone for now, and gives us time to
> think about the interface changes that may be needed long term, while
> still giving most of the operator-debug and usability/scalabilty
> benefits that I think folks pushing for Ansible are looking for.
>
> Steve
>
>
>
>
>> On Sat, Jul 8, 2017 at 12:20 AM, James Slagle 
>> wrote:
>>>
>>> On Fri, Jul 7, 2017 at 5:31 PM, David Moreau Simard 
>>> wrote:
>>> > On Fri, Jul 7, 2017 at 1:50 PM, James Slagle 
>>> > wrote:
>>> >> (0) tripleo-quickstart which follows the common and well accepted
>>> >> approach to bundling a set of Ansible playbooks/roles.
>>> >
>>> > I don't want to de-rail the thread but I really want to bring some
>>> > attention to a pattern that tripleo-quickstart has been using across
>>> > it's playbooks and roles.
>>> > I sincerely hope that we can find a better implementation should we
>>> > start developing new things from scratch.
>>>
>>> Yes, just to clarify...by "well accepted" I just meant how the git
>>> repo is organized and how you are expected to interface with those
>>> playbooks and roles as opposed to what those playbooks/roles actually
>>> do.
>>>
>>> > I'll sound like a broken record for those that have heard me mention
>>> > this before but for those that haven't, here's a concrete example of
>>> > how things are done today:
>>> > (Sorry for the link overload, making sure the relevant information is
>>> > available)
>>> >
>>> > For an example tripleo-quickstart job, here's the console [1] and it's
>>> > corresponding ARA report [2]:
>>> > - A bash script is created [3][4][5] from a jinja template [6]
>>> > - A task executes the bash script [7][8][9]
>>>
>>> From my limited experience, I believe the intent was that the
>>> playbooks should do what a user is expected to do so that it's as
>>> close to reproducing the user interface of TripleO 1:1.
>>>
>>> For example, we document users running commands from a shell prompt.
>>> Therefore, oooq ought to do the same thing as close as possible.
>>> Obviously there will be gaps, just as there is with tripleo.sh, but I
>>> feel that both tools (tripleo.sh/oooq) were trying to be faithful to
>>> our published docs as mush as possible, and I think there's something
>>> to be commended there.
>>>
>>> Not saying it's right or wong, just that I believe that was the intent.
>>>
>>> An alternative would be custom ansible modules that exposed tasks for
>>> interfacing with our API directly. That would also be valuable, as
>>> that code path is mostly untested now outside of the UI and CLI.
>>>
>>> I think that tripleo-quickstart is a slightly different class of
>>> "thing" from the other current Ansible uses I mentioned, in that it
>>> sits at a layer above everything else. It's meant to automate TripleO
>>> itself vs TripleO automating things. Regardless, we should certainly
>>> consider how it fits into a larger plan.
>>>
>>> --
>>> -- James Slagle
>>> --

[openstack-dev] [puppet] Meeting on July 11, 2017

2017-07-10 Thread Alex Schultz

Hey folks,

Just a heads up we have a meeting scheduled for tomorrow July 11, 2017
@ 1500 UTC.  The agenda is available here[0].  I would like to take a
few mins to review any outstanding work for Pike.  Feel free to add
any additional topics.

Thanks,
-Alex

[0] https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20170711

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo][all] next week is deadline for final release for non-client libraries

2017-07-10 Thread ChangBo Guo

OpenStackers,

According to Pike Schedule https://releases.openstack.org/pike/schedule.html

Jul 17 - Jul 21 is the deadline for final release for oslo libraries, so
please pay more attentions to your reviews which are needed for Pike. Feel
free to ping me if you want to quicken the review process.

-- 
ChangBo Guo(gcb)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tricircle]

2017-07-10 Thread meher.hihi

Hello everybody,

I posted before a problem related to installing the tricircle on a single node,
the script stopped with a keystone startup. You advised me to see the / etc /
apache2 / sites-enabled folder to see if the keystone config files are
included. But, I have not found this folder, yet the httpd service is properly
installed, the name of this file changes according to the distribution? I use
RHEL 7, thank you in advance!

Meher

[Logo Orange]

Meher Hihi
Intern
ORANGE/IMT/OLN/WTC/CMA/MAX
Fixe : +33 2 96 07 03
71
Mobile : +33 7 58 38 68
87
meher.h...@orange.com

De : HIHI Meher IMT/OLN
Envoyé : mercredi 28 juin 2017 15:12
À : 'openstack-dev@lists.openstack.org'
Objet : [openstack-dev][tricircle]

Hello everyone,

I introduce myself; Meher Hihi; I am doing my internship at Orange Labs
Networks Lannion-France for the diploma of computer network and
telecommunications engineer.

I am working on innovative distribution solutions for the virtualization
infrastructure of the network functions and more specifically on the Openstack
Tricircle solution, which is why I join your community to participate in your
discussions and learn from your advice.

Indeed, I try to install Tricircle on a single node by following this
documentation
"https://docs.openstack.org/developer/tricircle/installation-guide.html#single-pod-installation-with-devstack;.
I managed to install Devstack without any problems, but when I modify the
local.conf file by adding the Tricircle plugin integration and the HOST_IP, the
script does not want to work and stops on an error of Start of the Keystone
service.

I wanted to know if the problem is with my config file that is attached or I
lack other things to configure. You will also find in the file the IP address
of the machine.

I thank you in advance for the help you will bring me. Sincerely,

Best regards,

Meher

[Logo Orange]

Meher Hihi
Intern
ORANGE/IMT/OLN/WNI/ODIS/NAVI
Fixe : +33 2 96 07 03
71
Mobile : +33 7 58 38 68
87
meher.h...@orange.com

Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.

This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.

Re: [openstack-dev] [tripleo] weekly meetings on #tripleo

2017-07-10 Thread Brent Eagles

+1 for giving it a try.

On Wed, Jul 5, 2017 at 2:26 PM, Emilien Macchi  wrote:

> After reading http://lists.openstack.org/pipermail/openstack-dev/2017-
> June/118899.html
> - we might want to collect TripleO's community feedback on doing
> weekly meetings on #tripleo instead of #openstack-meeting-alt.
>
> I see some direct benefits:
> - if you come up late in meetings, you could easily read backlog in
> #tripleo
> - newcomers not aware about the meeting channel wouldn't have to search
> for it
> - meeting would maybe get more activity and we would expose the
> information more broadly
>
> Any feedback on this proposal is welcome before we make any change (or
> not).
>
> Thanks,
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Oslo] PTG Planning

2017-07-10 Thread ChangBo Guo

Hi Oslo folks,

I created a planning etherpad[1] for topics for the next PTG in Denver.
Please add any topics there,  then we can schedule the topics before the
PTG.

[1]https://etherpad.openstack.org/p/oslo-ptg-queens

-- 
ChangBo Guo(gcb)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] scheduling oslosphinx for retirement at the start of queens

2017-07-10 Thread ChangBo Guo

+1

2017-07-10 21:38 GMT+08:00 Davanum Srinivas :

> +1 from me
>
> On Mon, Jul 10, 2017 at 9:10 AM, Doug Hellmann 
> wrote:
> > Oslo team,
> >
> > With all documentation now moving to use the openstackdocs theme,
> > I propose that we retire the oslosphinx repository during Queens.
> > We should go ahead and create the stable/pike branch at the end of
> > this cycle, so that we have a way to deal with bugs in existing
> > pike releases, but I think we can retire the repository at any point
> > after that.
> >
> > Thoughts?
> >
> > Doug
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Davanum Srinivas :: https://twitter.com/dims
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
ChangBo Guo(gcb)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] scheduling oslosphinx for retirement at the start of queens

2017-07-10 Thread Davanum Srinivas

+1 from me

On Mon, Jul 10, 2017 at 9:10 AM, Doug Hellmann  wrote:
> Oslo team,
>
> With all documentation now moving to use the openstackdocs theme,
> I propose that we retire the oslosphinx repository during Queens.
> We should go ahead and create the stable/pike branch at the end of
> this cycle, so that we have a way to deal with bugs in existing
> pike releases, but I think we can retire the repository at any point
> after that.
>
> Thoughts?
>
> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] scheduling oslosphinx for retirement at the start of queens

2017-07-10 Thread Sean Dague

On 07/10/2017 09:10 AM, Doug Hellmann wrote:
> Oslo team,
> 
> With all documentation now moving to use the openstackdocs theme,
> I propose that we retire the oslosphinx repository during Queens.
> We should go ahead and create the stable/pike branch at the end of
> this cycle, so that we have a way to deal with bugs in existing
> pike releases, but I think we can retire the repository at any point
> after that.
> 
> Thoughts?

+1 and thanks for building the oslosphinx module years ago to start us
down this standardization on themes.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

2017-07-10 Thread Michele Baldessari

On Fri, Jul 07, 2017 at 10:39:25AM -0700, Emilien Macchi wrote:
> Alex has demonstrated high technical and community skills in TripleO -
> where he's already core on THT, instack-undercloud, and puppet-tripleo
> - but also very involved in other repos.
> I propose that we extend his core status to all TripleO projects and
> of course trust him (like we trust all core members) to review patches
> were we feel confortable with.
> 
> He has shown an high interest in reviewed other TripleO projects and I
> think he would be ready for this change.
> As usual, this is an open proposal, any feedback is welcome.

+1

-- 
Michele Baldessari
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Steven Hardy

On Sun, Jul 9, 2017 at 8:44 AM, Yolanda Robla Mota  wrote:
> What i'd like to dig more is how Ansible and Heat can live together. And
> what features do Heat offer that are not covered by Ansible as well? Is
> there still the need to have Heat as the main engine, or could that be
> replaced by Ansible totally in the future?

The main interface provided by Heat which AFAIK cannot currently be
replaced by Ansible is the parameters schema, where the template
parameters are exposed (that include description, type and constraint
data) in a format that is useful to e.g building the interfaces for
tripleo-ui

Ansible has a different approach to role/playbook parameters AFAICT,
which is more a global namespace with no type validation, no way to
include description data or tags with variable declarations, and no
way to specify constraints (other than perhaps hainvg custom modules
or playbook patterns that perform the validations early in the
deployment).

This is kind of similar to how the global namespace for hiera works
with our puppet model, although that at least has the advantage of
namespacing foo::something::variable, which again doesn't have a
direct equivalent in the ansible role model AFAIK (happy to be
corrected here, I'm not an ansible expert :)

For these reasons (as mentioned in my reply to James), I think a first
step of a "split stack" model where heat deploys the nodes/networks
etc, then outputs data that can be consumed by Ansible is reasonable -
it leaves the operator interfaces alone for now, and gives us time to
think about the interface changes that may be needed long term, while
still giving most of the operator-debug and usability/scalabilty
benefits that I think folks pushing for Ansible are looking for.

Steve




> On Sat, Jul 8, 2017 at 12:20 AM, James Slagle 
> wrote:
>>
>> On Fri, Jul 7, 2017 at 5:31 PM, David Moreau Simard 
>> wrote:
>> > On Fri, Jul 7, 2017 at 1:50 PM, James Slagle 
>> > wrote:
>> >> (0) tripleo-quickstart which follows the common and well accepted
>> >> approach to bundling a set of Ansible playbooks/roles.
>> >
>> > I don't want to de-rail the thread but I really want to bring some
>> > attention to a pattern that tripleo-quickstart has been using across
>> > it's playbooks and roles.
>> > I sincerely hope that we can find a better implementation should we
>> > start developing new things from scratch.
>>
>> Yes, just to clarify...by "well accepted" I just meant how the git
>> repo is organized and how you are expected to interface with those
>> playbooks and roles as opposed to what those playbooks/roles actually
>> do.
>>
>> > I'll sound like a broken record for those that have heard me mention
>> > this before but for those that haven't, here's a concrete example of
>> > how things are done today:
>> > (Sorry for the link overload, making sure the relevant information is
>> > available)
>> >
>> > For an example tripleo-quickstart job, here's the console [1] and it's
>> > corresponding ARA report [2]:
>> > - A bash script is created [3][4][5] from a jinja template [6]
>> > - A task executes the bash script [7][8][9]
>>
>> From my limited experience, I believe the intent was that the
>> playbooks should do what a user is expected to do so that it's as
>> close to reproducing the user interface of TripleO 1:1.
>>
>> For example, we document users running commands from a shell prompt.
>> Therefore, oooq ought to do the same thing as close as possible.
>> Obviously there will be gaps, just as there is with tripleo.sh, but I
>> feel that both tools (tripleo.sh/oooq) were trying to be faithful to
>> our published docs as mush as possible, and I think there's something
>> to be commended there.
>>
>> Not saying it's right or wong, just that I believe that was the intent.
>>
>> An alternative would be custom ansible modules that exposed tasks for
>> interfacing with our API directly. That would also be valuable, as
>> that code path is mostly untested now outside of the UI and CLI.
>>
>> I think that tripleo-quickstart is a slightly different class of
>> "thing" from the other current Ansible uses I mentioned, in that it
>> sits at a layer above everything else. It's meant to automate TripleO
>> itself vs TripleO automating things. Regardless, we should certainly
>> consider how it fits into a larger plan.
>>
>> --
>> -- James Slagle
>> --
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
>
> Yolanda Robla Mota
>
> Principal Software Engineer, RHCE
>
> Red Hat
>
> C/Avellana 213
>
> Urb Portugal
>
> yrobl...@redhat.comM: +34605641639
>
>
> __
> OpenStack

Re: [openstack-dev] [TripleO] Forming our plans around Ansible

2017-07-10 Thread Steven Hardy

On Fri, Jul 7, 2017 at 6:50 PM, James Slagle  wrote:
> I proposed a session for the PTG
> (https://etherpad.openstack.org/p/tripleo-ptg-queens) about forming a
> common plan and vision around Ansible in TripleO.
>
> I think it's important however that we kick this discussion off more
> broadly before the PTG, so that we can hopefully have some agreement
> for deeper discussions and prototyping when we actually meet in
> person.

Thanks for starting this James, it's a topic that I've also been
giving quite a lot of thought to lately (and as you've seen, have
pushed some related patches) so it's good to get some broader
discussions going.

> Right now, we have multiple uses of Ansible in TripleO:
>
> (0) tripleo-quickstart which follows the common and well accepted
> approach to bundling a set of Ansible playbooks/roles.

FWIW I agree with Giulio that quickstart is a separate case, and while
I also do agree with David that there's plenty of scope for
improvement of the oooq user experience, but I'm going to focus on the
TripleO deployment aspects below.

> (1) Mistral calling Ansible. This is the approach used by
> tripleo-validations where Mistral directly executes ansible playbooks
> using a dynamic inventory. The inventory is constructed from the
> server related stack outputs of the overcloud stack.
>
> (2) Ansible running playbooks against localhost triggered by the
> heat-config Ansible hook. This approach is used by
> tripleo-heat-templates for upgrade tasks and various tasks for
> deploying containers.
>
> (3) Mistral calling Heat calling Mistral calling Ansible. In this
> approach, we have Mistral resources in tripleo-heat-templates that are
> created as part of the overcloud stack and in turn, the created
> Mistral action executions run ansible. This has been prototyped with
> using ceph-ansible to install Ceph as part of the overcloud
> deployment, and some of the work has already landed. There are also
> proposed WIP patches using this approach to install Kubernetes.
>
> There are also some ideas forming around pulling the Ansible playbooks
> and vars out of Heat so that they can be rerun (or run initially)
> independently from the Heat SoftwareDeployment delivery mechanism:
>
> (4) https://review.openstack.org/#/c/454816/
>
> (5) Another idea I'd like to prototype is a local tool that runs on
> the undercloud and pulls all of the SoftwareDeployment data out of
> Heat as the stack is being created and generates corresponding Ansible
> playbooks to apply those deployments. Once a given playbook is
> generated by the tool, the tool would signal back to Heat that the
> deployment is complete. Heat then creates the whole stack without
> actually applying a single deployment to an overcloud node. At that
> point, Ansible (or Mistral->Ansible for an API) would be used to do
> the actual deployment of the Overcloud with the Undercloud as the
> ansible runner.

Yeah so my idea with (4), and subsequent patches such as[1] is to
gradually move the deploy steps performed to configure services (on
baremetal and in containers) to a single ansible playbook.

There's currently still heat orchestration around the host preparation
(although this is performed via ansible) and iteration over each step
(where we re-apply the same deploy-steps playbook with an incrementing
step variable, but this could be replaced by e.g an ansible or mistral
loop), but my idea was to enable end-to-end configuration of nodes via
ansible-playbook, without the need for any special tooks (e.g we
refactor t-h-t enough that we don't need any special tools, and we
make deploy-steps-playbook.yaml the only method of deployment (for
baremetal and container cases)

[1] https://review.openstack.org/#/c/462211/

> All of this work has merit as we investigate longer term plans, and
> it's all at different stages with some being for dev/CI (0), some
> being used already in production (1 and 2), some just at the
> experimental stage (3 and 4), and some does not exist other than an
> idea (5).

I'd like to get the remaining work for (4) done so it's a supportable
option for minor updates, but there's still a bit more t-h-t
refactoring required to enable it I think, but I think we're already
pretty close to being able to run end-to-end ansible for most of the
PostDeploy steps without any special tooling.

Note this related patch from Matthieu:

https://review.openstack.org/#/c/444224/

I think we'll need to go further here but it's a starting point which
shows how we could expose ansible tasks from the heat stack outputs as
a first step to enabling standalone configuration via ansible (or
mistral->ansible)

> My intent with this mail is to start a discussion around what we've
> learned from these approaches and start discussing a consolidated plan
> around Ansible. And I'm not saying that whatever we come up with
> should only use Ansible a certain way. Just that we ought to look at
> how users/operators interact with Ansible

[openstack-dev] [oslo] scheduling oslosphinx for retirement at the start of queens

Oslo team,

With all documentation now moving to use the openstackdocs theme,
I propose that we retire the oslosphinx repository during Queens.
We should go ahead and create the stable/pike branch at the end of
this cycle, so that we have a way to deal with bugs in existing
pike releases, but I think we can retire the repository at any point
after that.

Thoughts?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] index_footer in openstack logs

2017-07-10 Thread Wesley Hayutin

On Mon, Jul 10, 2017 at 2:24 AM, Andreas Jaeger  wrote:

> On 2017-07-10 05:03, Wesley Hayutin wrote:
> > posting to openstack-dev
> >
> > On Sun, Jul 9, 2017 at 10:08 AM, Wesley Hayutin  > > wrote:
> >
> > Greetings Andreas, Paul
> >
> > I'm looking for some pointers on how to include some instructions in
> > our openstack logs in the same way the devstack gate works [1] and
> > I'm not able to piece things together atm.
> >
> > I see the support for adding an index_footer was removed in [2], but
> > I don't see what it was replaced by.  I was hoping you guys could
> > point us in the right direction to enable embedding instructions
> > directly in our logs like [3]
> >
>
> Nope - we had *duplicate* macros, the support is still there. We removed
> one way of publishing that we never used.
>
> Do you know codesearch.openstack.org? Use it to find the code like
>
> http://codesearch.openstack.org/?q=Types%20of%20logs=nope==
>
>
> Andreas
>

Got it,
http://codesearch.openstack.org/?q=tempest-overview.html=nope==

I was not aware of codesearch.openstack

Thanks



>
> > Thank you for the help!
> >
> > [1] https://github.com/openstack-infra/devstack-gate/tree/
> master/help  master/help>
> > [2] https://github.com/openstack-infra/project-config/commit/
> 183aabbeaf528f5ef637a7bb51245eea4fab94b8#diff-
> 03d414c17dcd54548b8810c4a442b655
> >  183aabbeaf528f5ef637a7bb51245eea4fab94b8#diff-
> 03d414c17dcd54548b8810c4a442b655>
> > [3] http://logs.openstack.org/29/480429/1/check/gate-tempest-
> dsvm-neutron-full-ubuntu-xenial/76071e1/logs/
> >  dsvm-neutron-full-ubuntu-xenial/76071e1/logs/>
> >
> >
>
>
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

2017-07-10 Thread Jiří Stránský


On 7.7.2017 19:39, Emilien Macchi wrote:

Alex has demonstrated high technical and community skills in TripleO -
where he's already core on THT, instack-undercloud, and puppet-tripleo
- but also very involved in other repos.
I propose that we extend his core status to all TripleO projects and
of course trust him (like we trust all core members) to review patches
were we feel confortable with.

He has shown an high interest in reviewed other TripleO projects and I
think he would be ready for this change.
As usual, this is an open proposal, any feedback is welcome.

Thanks,



+1

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Bug reports and backporting

2017-07-10 Thread Rob Cresswell

Hey everyone!

Over the past few months there's been an increase in the number of patches 
without bug reports, and a decrease in the number of backports. I wanted to 
take a moment to explain why bug reports are useful, not just bureaucracy, and 
how they link to backports.

Bug reports help in a few ways:
- They can contain logs, images and discussion about the nature of a bug that 
helps reviewers do their job faster. This kind of verbosity is ideal for a bug 
report, but not suited to a commit message. Generally, the bug report should 
provide information about the *problem*, while the commit message provides 
information about the *solution*.
- For most people, bug reports are much easier to search than parsing git logs. 
Remember that not everyone is a git wizard.
- The tagging system allows us to mark things for backporting, with tags like 
"ocata-backport-potential". The tag system is currently how I manage most of 
our backports.
- Bugs that have been correctly closed (i.e. "Fix Released") and targeted to a 
milestone (i.e. "Pike-3") provide helpful metrics for how the project is 
progressing.

In most cases, a bug is *required* on a patch. If the patch is extremely small 
and self-explanatory - things like whitespace cleanup and typos - then we have 
no need to hold them up. Aside from that, I expect cores to please ask for bug 
reports to be added, and where necessary, target them to a milestone and add 
tags for service knowledge, backporting etc. Otherwise, I have to manually go 
through all of the recent merged changes looking for eligible backports, rather 
than just clicking the "ocata-backport-potential" tag.

Thanks,
Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

On Fri, Jul 7, 2017 at 1:39 PM, Emilien Macchi  wrote:
> Alex has demonstrated high technical and community skills in TripleO -
> where he's already core on THT, instack-undercloud, and puppet-tripleo
> - but also very involved in other repos.
> I propose that we extend his core status to all TripleO projects and
> of course trust him (like we trust all core members) to review patches
> were we feel confortable with.
>
> He has shown an high interest in reviewed other TripleO projects and I
> think he would be ready for this change.
> As usual, this is an open proposal, any feedback is welcome.

+1


-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][stable][ptls] Tagging mitaka as EOL

2017-07-10 Thread Andreas Jaeger

On 2017-07-10 13:33, Gary Kotton wrote:
> Hi,
> Will this also be moving to eol - 
> https://github.com/openstack/requirements/blob/stable/mitaka/global-requirements.txt?

No, not planned yet,

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

2017-07-10 Thread Brent Eagles

On Fri, Jul 7, 2017 at 3:09 PM, Emilien Macchi  wrote:

> Alex has demonstrated high technical and community skills in TripleO -
> where he's already core on THT, instack-undercloud, and puppet-tripleo
> - but also very involved in other repos.
> I propose that we extend his core status to all TripleO projects and
> of course trust him (like we trust all core members) to review patches
> were we feel confortable with.
>
> He has shown an high interest in reviewed other TripleO projects and I
> think he would be ready for this change.
> As usual, this is an open proposal, any feedback is welcome.
>
> Thanks,
> --
> Emilien Macchi
>
>
+1. Yes, please!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] bug triage experimentation

2017-07-10 Thread Sean Dague

On 07/05/2017 03:23 PM, Emilien Macchi wrote:

> 
> I also believe that some of the scripts could be transformed into
> native features of Storyboard where bugs could be auto-triaged
> periodically without human intervention.
> Maybe it would convince more OpenStack projects to leave Launchpad and
> adopt Storyboard?
> I would certainly one of those and propose such a change for TripleO &
> related projects.

Maybe... my concern there is that workflow encoded into trackers is
pretty static, and it's hard to evolve, because it impacts all users of
that platform. Where as a script that processes bugs externally can
adapt really quickly based on what's working / not working with a
particular team. There is no 1 right way to handle bugs, it's just about
making every bug handling team the most effective that they can be.
Which means I assume that different teams would find different parts of
this useful, and other parts things they wouldn't want to use at all.
That's why I tried to make every "processing unit" it's own cli.

Ideally storyboard would just be a lot more receptive to these kinds of
things, by emitting a more native event stream, and having really good
tag support (preferably actually project scoped tags, so setting it on
the nova task doesn't impact the neutron tasks on the same story, as an
for instance) so the hack we need to do on LP isn't needed. But,
actually, beyond that, keeping the processing logic team specific is a
good thing. It's much like the fact that we've largely done gerrit
review dashboards client side, because they are fast to iterate on, then
server side.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][stable][ptls] Tagging mitaka as EOL

2017-07-10 Thread Gary Kotton

Hi,
Will this also be moving to eol - 
https://github.com/openstack/requirements/blob/stable/mitaka/global-requirements.txt?
Thanks
Gary

On 7/10/17, 9:23 AM, "Andreas Jaeger"  wrote:

On 2017-07-07 17:37, Doug Szumski wrote:
> 
>> On 2017-07-05 12:13, Joshua Hesketh wrote:
>>> Hi all,
>>>
>>> Very sorry for the delay on processing this request. I have now EOL'd 
>>> stable/mitaka branches for projects listed in [1].
>>>
>>> If there are any mistakes it should be possible to restore the branch 
>>> at the correct position. Similarly please let me know if there were 
>>> any projects that should have been EOL'd but missed out.
> 
> Please also remove stable/mitaka from bareon-ironic

OK, added. I'm waiting for packstack confirmation.

The current list is now:


openstack/sahara-extra   stable/icehousePlease retire
openstack/sahara-extra   stable/mitaka  Please retire
openstack/packstack  stable/kiloWaiting for confirmation
openstack/packstack  stable/liberty Waiting for confirmation
openstack/packstack  stable/mitaka  Waiting for confirmation
openstack/bareon-ironic  stable/mitaka  Please retire

openstack/rpm-packaging  stable/mitaka  Please retire
openstack/training-labs  stable/mitaka  Please retire

Not done in

https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.githubusercontent.com_tbreeds_c99e62bf8da19380e4eb130be8783be7_raw_6d02deb40e07516ce8fc529d2ba8c74af11a5a6b_mitaka-5Feol-5Fdata.txt=DwIGaQ=uilaK90D4TOVoH58JNXRgQ=PMrZQUSXojEgJQPh7cZrz1Lvja0OwAstg0U82FalZrw=8JDeI5-ToCWlT4wlMOEK26WybSrP1mfZXQUx_wH4Yvo=y5Ib3yOnStYUMsnSZ62_ZUm0zbcLuqrk34Hfk1mVORA=
 

openstack/astara stable/mitaka  Please retire


Special treatment:
openstack/training-labs icehouse-eol (just delete branch, tag exists)
openstack/training-labs juno-eol (delete branch, create tag instead)


Any other late comers?

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [congress] Using congress to improve the consistency of configuration files.

2017-07-10 Thread valentin.matton


  
  
Hi Eric,
  
  Here is the blueprint : 
https://blueprints.launchpad.net/congress/+spec/configuration-files-validation
  
  As Pierre Crégut presented it in his reply to Clint, the use of
  Congress 
  we suggested and config management systems complement one another.
 Essentially, the latter reproduce delimited and likely
  valid configurations. 
  But there is little formal validation and you'd rely on tests as
  soon as you
  have to make a change. Tests should be used in validation but are
  not 
  best-suited to cover many constraints.

 By means of Congress, we could aggregate a lot of informal
  requirements 
  and rules to define what a valid state is, in a more manageable
  way. 
  We think that a declarative approach to the validation of
  configuration 
  files would be very fitting.
  
  We'd also like to discuss the prototype and how to improve its
  design with the Congress team.
  
  V. Matton.

Le 07/07/2017 à 01:16, Eric K a écrit :


  Hi Valentin,

Very cool to hear about your use case and vision! It definitely sounds
like the kind of use case Congress is well-equipped to solve using a
flexible, declarative rule language.

I'd love to explore the use case further (and where it fits along side
config management systems as Clint mentioned). I'm especially curious to
learn more about the prototype and see how I can be of help from Congress
team.

I did not see the blueprint link in the original message; missed paste
perhaps?

-Eric Kao (ekcs)



On 7/4/17, 6:29 AM, "valentin.mat...@orange.com"
 wrote:


  
We would like to use congress to check the consistency of the
configuration files used by the various Openstack services on different
nodes.

Although installers do a great job for ensuring that the initial
definition of those files are correct, it may be necessary to tweak
those files on running instances
or to use templates that are out of the bounds of the pre-configured
use-cases. Then the administrator must modify the configuration without
any safety net.

Congress is such a safety net but it ensures policies on live resources
deployed in the cloud, not on how the cloud is configured but we think
that it could be extended
to perform such verification with the adequate datasource.
So we propose a new datasource that will query each node to fetch its
configuration files as long as they follow oslo.config requirements and
structure.
As a first step we propose to use agents deployed on the different nodes
explicitly configured with the list of configuration files that push
those files to the central
congress service. Later on, oslo.config could be modified to provide a
hook used to push config files directly from running services.

The new datasource displays not only the option values, the file, host
where they are defined but also the associated metadata so that generic
rules can be defined.
It is then possible to define rules:
- that constrain the value of options between different nodes
- that constrain the values between different services or different
service plugins.
- it is even possible to use the knowledge of those configuration
options to check policies on live resources (for example when there is a
limitation or a bug in a given
driver).

We have a working prototype and the corresponding specification along
those principles that we would like to share. An initial blueprint has
been submitted here:
Please feel free to react

V. Matton

__
___

Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme
ou falsifie. Merci.

This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have
been modified, changed or falsified.
Thank you.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

  
  


__
OpenStack Development Mailing List

Re: [openstack-dev] [nova] bug triage experimentation

2017-07-10 Thread Sean Dague

On 07/05/2017 03:07 PM, Emilien Macchi wrote:
> On Wed, Jun 28, 2017 at 7:33 AM, Ben Nemec  wrote:
>>
>>
>> On 06/23/2017 11:52 AM, Sean Dague wrote:
>>>
>>> The Nova bug backlog is just over 800 open bugs, which while
>>> historically not terrible, remains too large to be collectively usable
>>> to figure out where things stand. We've had a few recent issues where we
>>> just happened to discover upgrade bugs filed 4 months ago that needed
>>> fixes and backports.
>>>
>>> Historically we've tried to just solve the bug backlog with volunteers.
>>> We've had many a brave person dive into here, and burn out after 4 - 6
>>> months. And we're currently without a bug lead. Having done a big giant
>>> purge in the past
>>>
>>> (http://lists.openstack.org/pipermail/openstack-dev/2014-September/046517.html)
>>> I know how daunting this all can be.
>>>
>>> I don't think that people can currently solve the bug triage problem at
>>> the current workload that it creates. We've got to reduce the smart
>>> human part of that workload.
>>>
>>> But, I think that we can also learn some lessons from what active github
>>> projects do.
>>>
>>> #1 Bot away bad states
>>>
>>> There are known bad states of bugs - In Progress with no open patch,
>>> Assigned but not In Progress. We can just bot these away with scripts.
>>> Even better would be to react immediately on bugs like those, that helps
>>> to train folks how to use our workflow. I've got some starter scripts
>>> for this up at - https://github.com/sdague/nova-bug-tools
>>
>>
>> Just saw the update on https://bugs.launchpad.net/nova/+bug/1698010 and I
>> don't agree that assigned but not in progress is an invalid state.  If it
>> persists for a period of time then sure, but to me assigning yourself a bug
>> is a signal that you're working on it and nobody else needs to. Otherwise
>> you end up with multiple people working a bug without realizing someone else
>> already was.  I've seen that happen more than once.
>>
>> Would it be possible to only un-assign such bugs if they've been in that
>> state for a week?  At that point it seems safe to say the assignee has
>> either moved on or that the bug is tricky and additional input would be
>> useful anyway.
>>
>> Otherwise, big +1 to a bug bot.  I need to try running it against the ~700
>> open TripleO bugs...
> 
> I just tried, please send complains to me if I broke something.
> 
> Sean, the tool is really awesome and I was wondering if we could move
> it to https://github.com/openstack-infra/release-tools so we
> centralize the tools.

Probably yes, eventually? Right now I'm honestly trying to figure out
the things that are useful here, and the ones that aren't, so a more
fluid experimentation is worth while.

My end game is to get some of these running and responding in real time
which means the core processing logic is going to evolve away from
scripts and into some kind of function plugin (the batch interface will
just still call them).

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] proposing Alex Schultz tripleo-core in all projects

thanks Alex!

+1

On 07/07/2017 07:39 PM, Emilien Macchi wrote:
> Alex has demonstrated high technical and community skills in TripleO -
> where he's already core on THT, instack-undercloud, and puppet-tripleo
> - but also very involved in other repos.
> I propose that we extend his core status to all TripleO projects and
> of course trust him (like we trust all core members) to review patches
> were we feel confortable with.
> 
> He has shown an high interest in reviewed other TripleO projects and I
> think he would be ready for this change.
> As usual, this is an open proposal, any feedback is welcome.


-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Forming our plans around Ansible