Re: [openstack-dev] [all] etcd3 as base service - update

2017-06-08 Thread Emilien Macchi
;>>>> Devstack installs etcd3 by default and points cinder to it:
>>>>> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
>>>>>
>>>>> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356
>>>>>
>>>>> Review in progress for keystone to use etcd3 for caching:
>>>>> https://review.openstack.org/#/c/469621/
>>>>>
>>>>> Doug is working on proposal(s) for oslo.config to store some
>>>>> configuration in etcd3:
>>>>> https://review.openstack.org/#/c/454897/
>>>>>
>>>>> So, feel free to turn on / test with etcd3 and report issues.
>>>>>
>>>>> Thanks,
>>>>> Dims
>>>>>
>>>>> --
>>>>> Davanum Srinivas :: https://twitter.com/dims
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

2017-06-08 Thread Emilien Macchi
On Thu, Jun 8, 2017 at 8:49 PM, Doug Hellmann <d...@doughellmann.com> wrote:
> Excerpts from Steven Dake (stdake)'s message of 2017-06-08 17:51:48 +:
>> Doug,
>>
>> In short, a configmap takes a bunch of config files, bundles them in a 
>> kubernetes object called a configmap, and then ships them to etcd.  When a 
>> pod is launched, the pod mounts the configmaps using tmpfs and the raw 
>> config files are available for use by the openstack services.
>
> That sounds like what confd does. Something puts data into one of
> several possible databases. confd takes it out and writes it to
> file(s) when the container starts. The app in the container reads
> the file(s).
>
> It sounds like configmaps would work well, too, it just doesn't
> sound like a fundamentally different solution.

Sorry for my lack of knowledge in ConfigMap but I'm trying to see how
we could bring pieces together.
Doug and I are currently investigating how oslo.config can be useful
to generate the parameters loaded by the application at startup,
without having to manage config with Puppet or Ansible.

If I understand correctly (and if not, please correct me, and maybe
propose something), we could use oslo.config to generate a portion of
ConfigMap (that can be imported in another ConfigMap iiuc) where we
would have parameters for one app.

Example with Keystone:

  apiVersion: v1
  kind: ConfigMap
  metadata:
name: keystone-config
namespace: DEFAULT
  data:
debug: true
rpc_backend: rabbit
... (parameters generated by oslo.config, and data fed by installers)

So iiuc we would give this file to k8s when deploying pods. Parameters
values would be automatically pushed into etcd, and used when
generating the configuration. Am I correct? (I need to understand if
we need to manually manage etcd key/values).

In that case, what deployments tools (like Kolla, TripleO, etc) would
expect from OpenStack to provide (tooling in oslo.config to generate
ConfigMap? etc.

Thanks for your help,

> Doug
>
>>
>> Operating on configmaps is much simpler and safer than using a different 
>> backing database for the configuration data.
>>
>> Hope the information helps.
>>
>> Ping me in #openstack-kolla if you have more questions.
>>
>> Regards
>> -steve
>>
>> -Original Message-
>> From: Doug Hellmann <d...@doughellmann.com>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>> <openstack-dev@lists.openstack.org>
>> Date: Thursday, June 8, 2017 at 10:12 AM
>> To: openstack-dev <openstack-dev@lists.openstack.org>
>> Subject: Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] 
>>[helm] Configuration management with etcd / confd
>>
>> Excerpts from Flavio Percoco's message of 2017-06-08 18:27:51 +0200:
>> > On 08/06/17 18:23 +0200, Flavio Percoco wrote:
>> > >On 07/06/17 12:04 +0200, Bogdan Dobrelya wrote:
>> > >>On 06.06.2017 18:08, Emilien Macchi wrote:
>> > >>>Another benefit is that confd will generate a configuration file 
>> when
>> > >>>the application will start. So if etcd is down *after* the app
>> > >>>startup, it shouldn't break the service restart if we don't ask 
>> confd
>> > >>>to re-generate the config. It's good for operators who were 
>> concerned
>> > >>>about the fact the infrastructure would rely on etcd. In that case, 
>> we
>> > >>>would only need etcd at the initial deployment (and during lifecycle
>> > >>>actions like upgrades, etc).
>> > >>>
>> > >>>The downside is that in the case of containers, they would still 
>> have
>> > >>>a configuration file within the container, and the whole goal of 
>> this
>> > >>>feature was to externalize configuration data and stop having
>> > >>>configuration files.
>> > >>
>> > >>It doesn't look a strict requirement. Those configs may (and should) 
>> be
>> > >>bind-mounted into containers, as hostpath volumes. Or, am I missing
>> > >>something what *does* make embedded configs a strict requirement?..
>> > >
>> > >mmh, one thing I liked about this effort was possibility of stop 
>> bind-mounting
>> > >config files into the containers. I'd rather find a way to not need 
>> any
>> > >bindmount and have the services get their configs themselves.
>> >
>> > Probably sent too early!

Re: [openstack-dev] [tripleo] [ci] Adding idempotency job on overcloud deployment.

2017-06-08 Thread Emilien Macchi
On Thu, Jun 8, 2017 at 1:47 PM, Sofer Athlan-Guyot <sathl...@redhat.com> wrote:
> Hi,
>
> Alex Schultz <aschu...@redhat.com> writes:
>
>> On Wed, Jun 7, 2017 at 5:20 AM, Sofer Athlan-Guyot <sathl...@redhat.com> 
>> wrote:
>>> Hi,
>>>
>>> Emilien Macchi <emil...@redhat.com> writes:
>>>
>>>> On Wed, Jun 7, 2017 at 12:45 PM, Sofer Athlan-Guyot <sathl...@redhat.com> 
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> I don't think we have such a job in place.  Basically that would check
>>>>> that re-running the "openstack deploy ..." command won't do anything.
>
> I've had a look at openstack-infra/tripleo-ci.  Should I test it in with
> ovb/quickstart or tripleo.sh.  Both way are fine by me, but I may be
> lacking context about which one is more relevant.
>
>>>>> We had such an error by the past[1], but I'm not sure this has been
>>>>> captured by an associated job.
>>>>>
>>>>> WDYT ?
>>>>
>>>> It would be interesting to measure how much time does it take to run
>>>> it again.
>>>
>>> Could you point out how such an experiment could be done ?
>>>
>>>> If it's short, we could add it to all our scenarios + ovb
>>>> jobs.  If it's long, maybe we need an additional job, but it would
>>>> take more resources, so maybe we could run it in periodic pipeline
>>>> (note that periodic jobs are not optimal since we could break
>>>> something quite easily).
>>>
>>> Just adding as context that the issue was already raised[1].  Beside
>>> time constraint, it was pointed out that we would also need to parse the
>>> log to find out if anything was restarted.  But it could be a second
>>> step.  For parsing, this code was pointed out[2].
>>>
>>
>> There's a few things that would need to be enabled in order to reuse
>> some of this work.  We'll need to add the ability to generate a report
>> on the puppet run[0]. And then we'll need to be able to capture it[1]
>> somewhere that we could then use that parsing code on.  From there,
>> just rerunning the installation would be a simple start to the
>> idempotency check.  In fuel, we had hacked in a special flag[2] that
>> we used in testing to actually rerun the task immediately to find when
>> a specific task was not idempotent in addition to also rerunning the
>> entire deployment. For tripleo a similar concept would be to rerun the
>> steps twice but that's usually not where the issues crop us for us. So
>> rerunning the entire installation deployment would be better as we
>> tend to have issues with configuration items between steps
>> conflicting.
>
> Maybe we could go with something equivalent to:
>
>   ts="$(date '+%F %T')"
>   ... re-run deploy command ...
>
>   sudo journalctl --since="${ts}" | egrep 'Stopping|Starting' | grep -v 
> 'user.*slice' > restarted.log
>   wc -l restarted.log
>
> This should be 0 on every overcloud nodes.
>
> This is simpler to implement and should catch any unwanted service
> restart.
>
> WDYT ?

It's smart, for services. It doesn't cover configuration files changes
and other resources managed by Puppet, like Keystone resources, etc.
But it's an excellent start to me.

>>
>> Thanks,
>> -Alex
>>
>> [0] https://review.openstack.org/#/c/273740/4/mcagents/puppetd.rb@204
>> [1] https://review.openstack.org/#/c/273740/4/mcagents/puppetd.rb@102
>> [2] https://review.openstack.org/#/c/273737/
>>
>>> [1] 
>>> http://lists.openstack.org/pipermail/openstack-dev/2017-March/114836.html
>>> [2] 
>>> https://review.openstack.org/#/c/279271/9/fuelweb_test/helpers/astute_log_parser.py@212
>>>
>>>>
>>>>> [1] https://bugs.launchpad.net/tripleo/+bug/1664650
>>>>> --
>>>>> Sofer Athlan-Guyot
>>>>>
>>>>> __
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>> --
>>>> Emilien Macchi
>>>>
>>>> __
>>>> OpenStack Development Mailing List (not for usage questions

Re: [openstack-dev] [tripleo][ironic] Hardware provisioning testing for Ocata

2017-06-08 Thread Emilien Macchi
On Thu, Jun 8, 2017 at 2:21 PM, Justin Kilpatrick <jkilp...@redhat.com> wrote:
> Morning everyone,
>
> I've been working on a performance testing tool for TripleO hardware
> provisioning operations off and on for about a year now and I've been
> using it to try and collect more detailed data about how TripleO
> performs in scale and production use cases. Perhaps more importantly
> YODA (Yet Openstack Deployment Tool, Another) automates the task
> enough that days of deployment testing is a set it and forget it
> operation.
>
> You can find my testing tool here [0] and the test report [1] has
> links to raw data and visualization. Just scroll down, click the
> capcha and click "go to kibana". I  still need to port that machine
> from my own solution over to search guard.
>
> If you have too much email to consider clicking links I'll copy the
> results summary here.
>
> TripleO inspection workflows have seen massive improvements from
> Newton with a failure rate for 50 nodes with the default workflow
> falling from 100% to <15%. Using patches slated for Pike that spurious
> failure rate reaches zero.
>
> Overcloud deployments show a significant improvement of deployment
> speed in HA and stack update tests.
>
> Ironic deployments in the overcloud allow the use of Ironic for bare
> metal scale out alongside more traditional VM compute. Considering a
> single conductor starts to struggle around 300 nodes it will be
> difficult to push a multi conductor setup to it's limits.
>
> Finally Ironic node cleaning, shows a similar failure rate to
> inspection and will require similar attention in TripleO workflows to
> become painless.
>
> [0] https://review.openstack.org/#/c/384530/
> [1] 
> https://docs.google.com/document/d/194ww0Pi2J-dRG3-X75mphzwUZVPC2S1Gsy1V0K0PqBo/
>
> Thanks for your time!

Hey Justin,

All of this is really cool. I was wondering if you had a list of bugs
that you've faced or reported yourself regarding to performances
issues in TripleO.
As you might have seen in a separate thread on openstack-dev, we're
planning a sprint on June 21/22th to improve performances in TripleO.
We would love your participation or someone from your team and if you
have time before, please add the deployment-time tag to the Launchpad
bugs that you know related to performances.

Thanks a lot,

> - Justin
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] A proposal for hackathon to reduce deploy time of TripleO

2017-06-08 Thread Emilien Macchi
On Thu, Jun 8, 2017 at 4:19 PM, Sagi Shnaidman <sshna...@redhat.com> wrote:
> Hi, all
>
> Thanks for your attention and proposals for this hackathon. With full
> understanding that optimization of deployment is on-going effort and should
> not be started and finished in these 2 days only, we still want to get focus
> on these issues in the sprint. Even if we don't solve immediately all
> problems, more people will be exposed to this field, additional tasks/bugs
> could be opened and scheduled, and maybe additional tests, process
> improvements and other insights will be introduced.
> If we don't reduce ci job time to 1 hour in Thursday it doesn't mean we
> failed the mission, please remember.
> The main goal of this sprint is to find problems and their work scope, and
> to find as many as possible solutions for them, using inter-team and team
> members collaboration and sharing knowledge. Ideally this collaboration and
> on-going effort will go further with such momentum. :)
>
> I suggest to do it in 21 - 22 Jun 2017 (Wednesday - Thursday). All other
> details are provided in etherpad:
> https://etherpad.openstack.org/p/tripleo-deploy-time-hack and in wiki as
> well: https://wiki.openstack.org/wiki/VirtualSprints
> We have a "deployment-time" tag for bugs:
> https://bugs.launchpad.net/tripleo/+bugs?field.tag=deployment-time Please
> use it for bugs that affect deployment time or CI job run time. It will be
> easier to handle them in the sprint.
>
> Please provide your comments and suggestions.

Thanks Sagi for bringing this up, this is really awesome.
One thing we could do to make this sprint productive is to report /
triage Launchpad bugs related to $topic so we have a list of things we
can work on during these 2 days.

Maybe we could go through:
https://launchpad.net/tripleo/+milestone/pike-2
https://launchpad.net/tripleo/+milestone/pike-3 and add
deployment-time to all the bugs we think it's related to performances.

Once we have the list, we'll work on them by priority and by area of knowledge.

Also, folks like face to face interactions. We'll take care of
preparing an open Bluejeans where folks can easily join and ask
questions. We'll probably be connected all day, so anyone can join
anytime. No schedule constraint here.

Any feedback is welcome,

Thanks!

> Thanks
>
>
>
> On Tue, May 23, 2017 at 1:47 PM, Sagi Shnaidman <sshna...@redhat.com> wrote:
>>
>> Hi, all
>>
>> I'd like to propose an idea to make one or two days hackathon in TripleO
>> project with main goal - to reduce deployment time of TripleO.
>>
>> - How could it be arranged?
>>
>> We can arrange a separate IRC channel and Bluejeans video conference
>> session for hackathon in these days to create a "presence" feeling.
>>
>> - How to participate and contribute?
>>
>> We'll have a few responsibility fields like tripleo-quickstart,
>> containers, storage, HA, baremetal, etc - the exact list should be ready
>> before the hackathon so that everybody could assign to one of these "teams".
>> It's good to have somebody in team to be stakeholder and responsible for
>> organization and tasks.
>>
>> - What is the goal?
>>
>> The goal of this hackathon to reduce deployment time of TripleO as much as
>> possible.
>>
>> For example part of CI team takes a task to reduce quickstart tasks time.
>> It includes statistics collection, profiling and detection of places to
>> optimize. After this tasks are created, patches are tested and submitted.
>>
>> The prizes will be presented to teams which saved most of time :)
>>
>> What do you think?
>>
>> Thanks
>> --
>> Best regards
>> Sagi Shnaidman
>
>
>
>
> --
> Best regards
> Sagi Shnaidman



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack-operators] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

2017-06-07 Thread Emilien Macchi
On Wed, Jun 7, 2017 at 3:31 PM, Doug Hellmann <d...@doughellmann.com> wrote:
>
> On Jun 7, 2017, at 7:20 AM, Emilien Macchi <emil...@redhat.com> wrote:
>
> On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <emil...@redhat.com> wrote:
>
> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it:
> https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>
>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.
>
>
> I'm also wondering if we could use oslo-config-generate directly to
> generate confd templates, with a new format. So we would have ini,
> yaml, json and confd.
> "confd" format would be useful when building rpms that we ship in
> containers.
> "yaml" format would be useful for installers to expose the options
> directly to the User Interface, so we know which params OpenStack
> provide and we could re-use the data to push it into etcd.
>
> Would it make sense?
>
>
> I did think about making oslo-config-generator also take the YAML file as
> input instead of scanning plugins, and then including all the output formats
> in the single command. I haven’t looked to see how much extra complexity
> that would add.

Do you mean taking the YAML file that we generate with Ben's work
(which would include the parameters values, added by some other
tooling maybe)?

I see 2 options at least:

* Let installers to feed etcd with the parameters by using this etcd
namespace $CUSTOM_PREFIX + /project/section/parameter (example
/node1/keystone/DEFAULT/debug).
  And patch oslo.config to be able to generate confd templates with
all the options (and ship the template in the package)
  I like this option because it provides a way for operators to learn
about all possible options in the configuration, with documentation
and default values.

* Also let installers to feed etcd but use a standard template like
you showed me last week (credits to you for the code):
http://paste.openstack.org/show/2KZUQsWYpgrcG2K8TDcE/
   I like this option because nothing has to be done in oslo.config,
since we use a standard template for all OpenStack configs (see the
paste ^)

Thoughts?

>
> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.
>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.
> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.
>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose para

Re: [openstack-dev] [tripleo] [ci] Adding idempotency job on overcloud deployment.

2017-06-07 Thread Emilien Macchi
On Wed, Jun 7, 2017 at 1:20 PM, Sofer Athlan-Guyot <sathl...@redhat.com> wrote:
> Hi,
>
> Emilien Macchi <emil...@redhat.com> writes:
>
>> On Wed, Jun 7, 2017 at 12:45 PM, Sofer Athlan-Guyot <sathl...@redhat.com> 
>> wrote:
>>> Hi,
>>>
>>> I don't think we have such a job in place.  Basically that would check
>>> that re-running the "openstack deploy ..." command won't do anything.
>>>
>>> We had such an error by the past[1], but I'm not sure this has been
>>> captured by an associated job.
>>>
>>> WDYT ?
>>
>> It would be interesting to measure how much time does it take to run
>> it again.
>
> Could you point out how such an experiment could be done ?

in openstack-infra/tripleo-ci, you would run another time the
overcloud deploy command. That's it.

>> If it's short, we could add it to all our scenarios + ovb
>> jobs.  If it's long, maybe we need an additional job, but it would
>> take more resources, so maybe we could run it in periodic pipeline
>> (note that periodic jobs are not optimal since we could break
>> something quite easily).
>
> Just adding as context that the issue was already raised[1].  Beside
> time constraint, it was pointed out that we would also need to parse the
> log to find out if anything was restarted.  But it could be a second
> step.  For parsing, this code was pointed out[2].
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/114836.html
> [2] 
> https://review.openstack.org/#/c/279271/9/fuelweb_test/helpers/astute_log_parser.py@212
>
>>
>>> [1] https://bugs.launchpad.net/tripleo/+bug/1664650
>>> --
>>> Sofer Athlan-Guyot
>>>
>>> __________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> Emilien Macchi
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> --
> Sofer Athlan-Guyot
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

2017-06-07 Thread Emilien Macchi
On Tue, Jun 6, 2017 at 6:08 PM, Emilien Macchi <emil...@redhat.com> wrote:
> Following-up the session that we had in Boston:
> https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
>
> Here's an update on where we are and what is being done.
>
>
> == Machine Readable Sample Config
>
> Ben's spec has been merged: https://review.openstack.org/#/c/440835/
> And also the code which implements it: 
> https://review.openstack.org/#/c/451081/
> He's now working on the documentation on how to use the feature.
>
> $ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml
>
> Here's an example of the output for Keystone config: https://clbin.com/EAfFM
> This feature was asked at the PTG, and it's already done!
>
>
> == Pluggable drivers for oslo.config
>
> Doug's spec has been well written and the feedback from Summit and the
> review was taken in account: https://review.openstack.org/#/c/454897/
> It's now paused because we think we could use confd (with etcd driver)
> to generate configuration files.
>
> Imagine the work done by Ben in Machine Readable Sample Config, that
> would allow us to generate Confd templates for all services (Keystone,
> Nova, etc) via a tool provided in oslo.config with all the options
> available for a namespace.

I'm also wondering if we could use oslo-config-generate directly to
generate confd templates, with a new format. So we would have ini,
yaml, json and confd.
"confd" format would be useful when building rpms that we ship in containers.
"yaml" format would be useful for installers to expose the options
directly to the User Interface, so we know which params OpenStack
provide and we could re-use the data to push it into etcd.

Would it make sense?

> We could have packaging builds (e.g. RDO distgit) using the tooling
> when building packages so we could ship confd templates in addition of
> ini configuration files.
> When services would start (e.g. in containers), confd would generate
> configuration files from the templates that is part of the container,
> and read the values from etcd.
>
> The benefit of doing this, is that a very little work is required in
> oslo.config to make this happen (only a tool to generate confd
> templates). It could be a first iteration.
> Another benefit is that confd will generate a configuration file when
> the application will start. So if etcd is down *after* the app
> startup, it shouldn't break the service restart if we don't ask confd
> to re-generate the config. It's good for operators who were concerned
> about the fact the infrastructure would rely on etcd. In that case, we
> would only need etcd at the initial deployment (and during lifecycle
> actions like upgrades, etc).
>
> The downside is that in the case of containers, they would still have
> a configuration file within the container, and the whole goal of this
> feature was to externalize configuration data and stop having
> configuration files.
>
>
> == What's next
>
> I see 2 short-term actions that we can work on:
>
> 1) Decide if whether or not confd solution would be acceptable for a
> start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
> be willing to use this feature. I'm also asking operators to give
> feedback on it.
>
> 2) Investigate how to expose parameters generated by Ben's work on
> Machine Readable Sample Config to the users (without having to
> manually maintain all options) - I think this has to be solved on the
> installers side, but I might be wrong; and also investigate how to
> populate parameters data into etcd. This tool could be provided by
> oslo.config probably.
>
>
>
> Any feedback from folks working on installers or from operators would
> be more than welcome!
>
> Thanks,
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] [ci] Adding idempotency job on overcloud deployment.

2017-06-07 Thread Emilien Macchi
On Wed, Jun 7, 2017 at 12:45 PM, Sofer Athlan-Guyot <sathl...@redhat.com> wrote:
> Hi,
>
> I don't think we have such a job in place.  Basically that would check
> that re-running the "openstack deploy ..." command won't do anything.
>
> We had such an error by the past[1], but I'm not sure this has been
> captured by an associated job.
>
> WDYT ?

It would be interesting to measure how much time does it take to run
it again. If it's short, we could add it to all our scenarios + ovb
jobs. If it's long, maybe we need an additional job, but it would take
more resources, so maybe we could run it in periodic pipeline (note
that periodic jobs are not optimal since we could break something
quite easily).

> [1] https://bugs.launchpad.net/tripleo/+bug/1664650
> --
> Sofer Athlan-Guyot
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] improving logs / config path in oooq-container jobs

2017-06-07 Thread Emilien Macchi
By reviewing https://review.openstack.org/#/c/447176 which is solving
an urgent issue in Infra to save space in logs server, I found out
that container jobs store config and logs in paths that are not easy
to find by newcomers.

Example: 
http://logs.openstack.org/76/447176/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/4e266e7/logs/oooq/overcloud-controller-0/var/log/config-data/
Folks have to go on
logs/oooq/overcloud-controller-0/var/log/config-data/ to find out
OpenStack configs on each containers.

I think we could make this path easier in our log collection scripts.
Also, we already mentioned that but it would be great to engage some
work to have a landing page when we click on logs, like devstack is
doing, with some basic documentation on where we can find which files
(logs, configs, etc).

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Deprecated Parameters Warning

2017-06-06 Thread Emilien Macchi
On Tue, Jun 6, 2017 at 6:53 AM, Saravanan KR <skram...@redhat.com> wrote:
> Hello,
>
> I am working on a patch [1] to list the deprecated parameters of the
> current plan. It depends on a heat patch[2] which provides
> parameter_group support for nested stacks. The change is to add a new
> workflow to analyze the plan templates and find out the list of
> deprecated parameters, identified by parameter_groups with label
> "deprecated".
>
> This workflow can be used by CLI and UI to provide a warning to the
> user about the deprecated parameters. This is only the listing,
> changes are required in tripleoclient to invoke and and provide
> warning. I am sending this mail to update the group, to bring
> awareness on the parameter deprecation.

I find this feature very helpful, specially with all the THT
parameters that we have and that are moving quite fast over the
cycles.
Thanks for working on it!

> Regards,
> Saravanan KR
>
> [1] https://review.openstack.org/#/c/463949/
> [2] https://review.openstack.org/#/c/463941/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [deployment] [oslo] [ansible] [tripleo] [kolla] [helm] Configuration management with etcd / confd

2017-06-06 Thread Emilien Macchi
Following-up the session that we had in Boston:
https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management

Here's an update on where we are and what is being done.


== Machine Readable Sample Config

Ben's spec has been merged: https://review.openstack.org/#/c/440835/
And also the code which implements it: https://review.openstack.org/#/c/451081/
He's now working on the documentation on how to use the feature.

$ oslo-config-generator --namespace keystone --formal yaml > keystone.yaml

Here's an example of the output for Keystone config: https://clbin.com/EAfFM
This feature was asked at the PTG, and it's already done!


== Pluggable drivers for oslo.config

Doug's spec has been well written and the feedback from Summit and the
review was taken in account: https://review.openstack.org/#/c/454897/
It's now paused because we think we could use confd (with etcd driver)
to generate configuration files.

Imagine the work done by Ben in Machine Readable Sample Config, that
would allow us to generate Confd templates for all services (Keystone,
Nova, etc) via a tool provided in oslo.config with all the options
available for a namespace.
We could have packaging builds (e.g. RDO distgit) using the tooling
when building packages so we could ship confd templates in addition of
ini configuration files.
When services would start (e.g. in containers), confd would generate
configuration files from the templates that is part of the container,
and read the values from etcd.

The benefit of doing this, is that a very little work is required in
oslo.config to make this happen (only a tool to generate confd
templates). It could be a first iteration.
Another benefit is that confd will generate a configuration file when
the application will start. So if etcd is down *after* the app
startup, it shouldn't break the service restart if we don't ask confd
to re-generate the config. It's good for operators who were concerned
about the fact the infrastructure would rely on etcd. In that case, we
would only need etcd at the initial deployment (and during lifecycle
actions like upgrades, etc).

The downside is that in the case of containers, they would still have
a configuration file within the container, and the whole goal of this
feature was to externalize configuration data and stop having
configuration files.


== What's next

I see 2 short-term actions that we can work on:

1) Decide if whether or not confd solution would be acceptable for a
start. I'm asking Kolla, TripleO, Helm, Ansible projects if they would
be willing to use this feature. I'm also asking operators to give
feedback on it.

2) Investigate how to expose parameters generated by Ben's work on
Machine Readable Sample Config to the users (without having to
manually maintain all options) - I think this has to be solved on the
installers side, but I might be wrong; and also investigate how to
populate parameters data into etcd. This tool could be provided by
oslo.config probably.



Any feedback from folks working on installers or from operators would
be more than welcome!

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] overcloud containers patches todo

2017-06-05 Thread Emilien Macchi
On Mon, Jun 5, 2017 at 2:34 AM, Dan Prince <dpri...@redhat.com> wrote:
> Hi,
>
> Any help reviewing the following patches for the overcloud
> containerization effort in TripleO would be appreciated:
>
> https://etherpad.openstack.org/p/tripleo-containers-todo

Nice summary, it really helps. Thanks.
Maybe could we prioritize the items that need to be reviewed asap,
just in case this is required for you to make faster progress.

> If you've got new services related to the containerization efforts feel
> free to add them here too.
>
> Thanks,
>
> Dan
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] overcloud containers patches todo

2017-06-05 Thread Emilien Macchi
On Mon, Jun 5, 2017 at 8:59 AM, Sagi Shnaidman <sshna...@redhat.com> wrote:
> Hi
> I think a "deep dive" about containers in TripleO and some helpful
> documentation would help a lot for valuable reviews of these container
> patches. The knowledge gap that's accumulated here is pretty big.

This is not the first time I'm hearing this, indeed it would be super useful.

> Thanks
>
> On Jun 5, 2017 03:39, "Dan Prince" <dpri...@redhat.com> wrote:
>>
>> Hi,
>>
>> Any help reviewing the following patches for the overcloud
>> containerization effort in TripleO would be appreciated:
>>
>> https://etherpad.openstack.org/p/tripleo-containers-todo
>>
>> If you've got new services related to the containerization efforts feel
>> free to add them here too.
>>
>> Thanks,
>>
>> Dan
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] tempest failures when deploying neutron-server in wsgi with apache

2017-06-02 Thread Emilien Macchi
On Thu, Jun 1, 2017 at 10:28 PM, Morales, Victor
<victor.mora...@intel.com> wrote:
> Hi Emilien,
>
> I noticed that the configuration file was created using puppet.  I submitted 
> a patch[1] that was targeting to include the changes in Devstack. My major 
> concern is with the value of WSGIScriptAlias which should be pointing to WSGI 
> script.

Thanks for looking, the script that is used is from
/usr/bin/neutron-api which is I think correct. If you look at logs,
you can see that API actually works but some tempest tests fail
though...

> Regards/Saludos
> Victor Morales
>
> [1] https://review.openstack.org/#/c/439191
>
> On 5/31/17, 4:40 AM, "Emilien Macchi" <emil...@redhat.com> wrote:
>
> Hey folks,
>
> I've been playing with deploying Neutron in WSGI with Apache and
> Tempest tests fail on spawning Nova server when creating Neutron
> ports:
> 
> http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/console.html#_2017-05-30_13_09_22_715400
>
> I haven't found anything useful in neutron-server logs:
> 
> http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/logs/apache/neutron_wsgi_access_ssl.txt.gz
>
> Before I file a bug in neutron, can anyone look at the logs with me
> and see if I missed something in the config:
> 
> http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/logs/apache_config/10-neutron_wsgi.conf.txt.gz
>
> Thanks for the help,
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tc][ptls][all] Potential Queens Goal: Move policy and policy docs into code

2017-06-02 Thread Emilien Macchi
On Thu, Jun 1, 2017 at 7:54 PM, Lance Bragstad <lbrags...@gmail.com> wrote:
> Hi all,
>
> I've proposed a community-wide goal for Queens to move policy into code and
> supply documentation for each policy [0]. I've included references to
> existing documentation and specifications completed by various projects and
> attempted to lay out the benefits for both developers and operators.
>
> I'd greatly appreciate any feedback or discussion.

I would rewrite what I said in the patch, but thanks for proposing it.
I like this one in a sense it will help our operators to control
policies and make their configuration easier and more consistent.

> Thanks!
>
> Lance
>
>
> [0] https://review.openstack.org/#/c/469954/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ptg] Strawman Queens PTG week slicing

2017-06-02 Thread Emilien Macchi
On Thu, Jun 1, 2017 at 4:38 PM, Thierry Carrez <thie...@openstack.org> wrote:
> Thierry Carrez wrote:
>> In a previous thread[1] I introduced the idea of moving the PTG from a
>> purely horizontal/vertical week split to a more
>> inter-project/intra-project activities split, and the initial comments
>> were positive.
>>
>> We need to solidify how the week will look like before we open up
>> registration (first week of June), so that people can plan their
>> attendance accordingly. Based on the currently-signed-up teams and
>> projected room availability, I built a strawman proposal of how that
>> could look:
>>
>> https://docs.google.com/spreadsheets/d/1xmOdT6uZ5XqViActr5sBOaz_mEgjKSCY7NEWcAEcT-A/pubhtml?gid=397241312=true
>
> OK, it looks like the feedback on this strawman proposal was generally
> positive, so we'll move on with this.
>
> For teams that are placed on the Wednesday-Friday segment, please let us
> know whether you'd like to make use of the room on Friday (pick between
> 2 days or 3 days). Note that it's not a problem if you do (we have space
> booked all through Friday) and this can avoid people leaving too early
> on Thursday afternoon. We just need to know how many rooms we might be
> able to free up early.

For TripleO, Friday would be good (at least the morning) but I also
think 2 days would be enough in case we don't have enough space.

- So let's book Wednesday / Thursday / Friday.
- We probably won't have anything on Friday afternoon, since I expect
people traveling usually at this time.
- If not enough room, no worries, we can have Wednesday / Thursday
only, we'll survive for sure.

Thanks,

> In the same vein, if your team (or workgroup, or inter-project goal) is
> not yet listed and you'd like to have a room in Denver, let us know ASAP.
>
> --
> Thierry Carrez (ttx)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [puppet][release][Release-job-failures] Tag of openstack/puppet-nova failed

2017-06-01 Thread Emilien Macchi
Thanks Doug for this one!

---
Emilien Macchi

On Jun 1, 2017 6:21 PM, "Doug Hellmann" <d...@doughellmann.com> wrote:

> Excerpts from jenkins's message of 2017-05-31 20:26:33 +:
> > Build failed.
> >
> > - puppet-nova-releasenotes http://logs.openstack.org/d9/
> d913ccd1ea88f3661c32b0fcfdac58d749cd4eb2/tag/puppet-nova-
> releasenotes/cefa30a/ : FAILURE in 2m 13s
> >
>
> This failure only prevented the release notes from being published, and
> did not block the actual release.
>
> The problem should be fixed by https://review.openstack.org/469872
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [EXTERNAL] Re: [Tripleo] deploy software on Openstack controller on the Overcloud

2017-06-01 Thread Emilien Macchi
On Thu, Jun 1, 2017 at 3:47 PM, Abhishek Kane <abhishek.k...@veritas.com> wrote:
> Hi Emilien,
>
> The bin does following things on controller:
> 1. Install core HyperScale packages.

Should be done by Puppet, with Package resource.

> 2. Start HyperScale API server

Should be done by Puppet, with Service resource.

> 3. Install UI packages. This will add new files to and modify some existing 
> files of Horison.

Should be done by Puppet, with Package resource and also some changes
in puppet-horizon maybe if you need to change Horizon config.

> 4. Create HyperScale user in mysql db. Create database and dump config. Add 
> permissions of nova and cinder DB to HyperScale user.

We have puppet-openstacklib which already manages DBs, you could
easily re-use it. Please look at puppet-nova for example to see how
things works in nova::db::mysql, etc.

> 5. Add ceilometer pollsters for additional stats and modify ceilometer files.

puppet-ceilometer I guess. What do you mean by "files"? Config files?

> 6. Change OpenStack configuration:
> a. Create rabbitmq exchanges

puppet-* modules already does it.

> b. Create keystone user

puppet-keystone already does it.

> c. Define new flavors

puppet-nova can manage flavors.

> d. Create HyperScale volume type and set default volume type to HyperScale in 
> cinder.conf.

we already support multi backends in tripleo, HyperScale would just be
a new addition. Re-use the bits please: puppet-cinder and
puppet-tripleo will need to be patched.

> e. Restart openstack’s services

Already done by openstack/puppet-* modules.

> 7. Configure HyperScale services

Should be done by your module, (you can either write a _config
provider if it's ini standard otherwise just do a template that you
ship in the module, like puppet-horizon).

> Once the controller is configured, we use HyperScale’s CLI to configure data 
> and compute nodes-
>
> On data node (cinder):
> 1. Install HyperScale data node packages.

Should be done by Puppet, with Package resource.

> 2. Change cinder.conf to add backend and change rpc_backend.

puppet-cinder

> 3. Give the raw data disks and meta disks to HyperScale storage layer for 
> usage.

what does it means? Do you run a CLI for that?

> 4. Configure HyperScale services.

Should be done by Puppet, with Service resource.

> 5. Dump config in the HyperScale database.

can be done by a script maybe?

>
> On compute (nova):
> 1. Install HyperScale compute packages.

Should be done by Puppet, with Package resource.

> 2. Configure cgroup.

we don't manage cgroups in TripleO AFIK yet but it's something we
could add, maybe with a puppet module.

> 3. Disable selinux.

Please don't do that. Disabling SElinux is a NOGO when adding new
features (sorry to care about Security).

> 4. Add ceilometer pollsters for additional stats and modify ceilometer files.

puppet-ceilometer

> 5. Modify qemu.conf to relax ACS checks.

puppet-nova maybe, but not sure we really want to do that:
https://vfio.blogspot.fr/2014/08/iommu-groups-inside-and-out.html

Any details on why you're doing it?

> 6. Modify libvirt.conf and libvirtd to allow live migration.

It's already supported by puppet-nova.

> 7. Change network settings.

Should be done by os-net-config in TripleO.

> 8. Configure HyperScale services.

Done by your module (again).

> 9. Dump config in the HyperScale database.

same as before.

>
> We assume that we will not require steps to install packages if we put 
> packages in the overcloud image. We have started to convert the bin and the 
> CLI into puppet modules.
>
>
> Regards,
> Abhishek

Hope it helped.

> On 6/1/17, 4:24 AM, "Emilien Macchi" <emil...@redhat.com> wrote:
>
> On Wed, May 31, 2017 at 6:29 PM, Dnyaneshwar Pawar
> <dnyaneshwar.pa...@veritas.com> wrote:
> > Hi Alex,
> >
> > Currently we have puppet modules[0] to configure our software which has
> > components on Openstack Controller, Cinder node and Nova node.
> > As per document[1] we successfully tried out role specific 
> configuration[2].
> >
> > So, does it mean that if we have an overcloud image with our packages
> > inbuilt and we call our configuration scripts using role specific
> > configuration, we may not need puppet modules[0] ? Is it acceptable
> > deployment method?
>
> So running a binary from Puppet, to make configuration management is
> not something we recommend.
> Puppet has been good at managing configuration files and services for
> example. In your module, you just manage a file and execute it. The
> problem with that workflow is we have no idea what happens in backend.
> Also we have no way to make Puppet run idempotent, which is one

Re: [openstack-dev] [tc][ptl][all] Potential Queens Goal: Continuing Python 3.5+ Support

2017-06-01 Thread Emilien Macchi
On Wed, May 31, 2017 at 10:38 PM, Mike <thin...@gmail.com> wrote:
> Hello everyone,
>
> For this thread we will be discussing continuing Python 3.5+ support.
> Emilien who has been helping with coordinating our efforts here with
> Pike can probably add more here, but glancing at our goals document
> [1] it looks like we have a lot of unanswered projects’ status, but
> mostly we have python 3.5 unit test voting jobs done thanks to this
> effort! I have no idea how to use the graphite dashboard, but here’s a
> graph [2] showing success vs failure with python-35 jobs across all
> projects.

Indeed, nice work from the community to make progress on this effort.

> Glancing at that I think it’s safe to say we can start discussions on
> moving forward with having our functional tests support python 3.5.
> Some projects are already ahead in this. Let the discussions begin so
> we can aid the decision in the  TC deciding our community wide goals
> for Queens [3].

+1 - making progress on functional tests looks like the next thing and
Queens cycle could be used. I'm happy to keep helping on coordination
if needed.

>
> [1] - https://governance.openstack.org/tc/goals/pike/python35.html
> [2] - 
> http://graphite.openstack.org/render/?width=1273=554&_salt=1496261911.56=00%3A00_20170401=23%3A59_20170531=sumSeries(stats.zuul.pipeline.gate.job.gate-*-python35.SUCCESS)=sumSeries(stats.zuul.pipeline.gate.job.gate-*-python35.FAILURE)
> [3] - https://governance.openstack.org/tc/goals/index.html
>
> —
> Mike Perez



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Tripleo] deploy software on Openstack controller on the Overcloud

2017-05-31 Thread Emilien Macchi
On Wed, May 31, 2017 at 6:29 PM, Dnyaneshwar Pawar
<dnyaneshwar.pa...@veritas.com> wrote:
> Hi Alex,
>
> Currently we have puppet modules[0] to configure our software which has
> components on Openstack Controller, Cinder node and Nova node.
> As per document[1] we successfully tried out role specific configuration[2].
>
> So, does it mean that if we have an overcloud image with our packages
> inbuilt and we call our configuration scripts using role specific
> configuration, we may not need puppet modules[0] ? Is it acceptable
> deployment method?

So running a binary from Puppet, to make configuration management is
not something we recommend.
Puppet has been good at managing configuration files and services for
example. In your module, you just manage a file and execute it. The
problem with that workflow is we have no idea what happens in backend.
Also we have no way to make Puppet run idempotent, which is one
important aspect in TripleO.

Please tell us what does the binary, and maybe we can convert the
tasks into Puppet resources that could be managed by your module. Also
make the resources by class (service), so we can plug it into the
composable services in TripleO.

Thanks,

> [0] https://github.com/abhishek-kane/puppet-veritas-hyperscale
> [1]
> https://docs.openstack.org/developer/tripleo-docs/advanced_deployment/node_config.html
> [2] http://paste.openstack.org/show/66/
>
> Thanks,
> Dnyaneshwar
>
> On 5/30/17, 6:52 PM, "Alex Schultz" <aschu...@redhat.com> wrote:
>
> On Mon, May 29, 2017 at 5:05 AM, Dnyaneshwar Pawar
> <dnyaneshwar.pa...@veritas.com> wrote:
>
> Hi,
>
> I am tying to deploy a software on openstack controller on the overcloud.
> One way to do this is by modifying ‘overcloud image’ so that all packages of
> our software are added to image and then run overcloud deploy.
> Other way is to write heat template and puppet module which will deploy the
> required packages.
>
> Question: Which of above two approaches is better?
>
> Note: Configuration part of the software will be done via separate heat
> template and puppet module.
>
>
> Usually you do both.  Depending on how the end user is expected to
> deploy, if they are using the TripleoPackages service[0] in their
> role, the puppet installation of the package won't actually work (we
> override the package provider to noop) so it needs to be in the
> images.  That being said, usually there is also a bit of puppet that
> needs to be written to configure the end service and as a best
> practice (and for development purposes), it's a good idea to also
> capture the package in the manifest.
>
> Thanks,
> -Alex
>
> [0]
> https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/tripleo-packages.yaml
>
>
> Thanks and Regards,
> Dnyaneshwar
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements][mistral][tripleo][horizon][nova][releases] release models for projects tracked in global-requirements.txt

2017-05-31 Thread Emilien Macchi
On Tue, May 30, 2017 at 11:11 PM, Matthew Thode
<prometheanf...@gentoo.org> wrote:
> On 05/30/2017 04:08 PM, Emilien Macchi wrote:
>> On Tue, May 30, 2017 at 8:36 PM, Matthew Thode
>> <prometheanf...@gentoo.org> wrote:
>>> We have a problem in requirements that projects that don't have the
>>> cycle-with-intermediary release model (most of the cycle-with-milestones
>>> model) don't get integrated with requirements until the cycle is fully
>>> done.  This causes a few problems.
>>>
>>> * These projects don't produce a consumable release for requirements
>>> until end of cycle (which does not accept beta releases).
>>>
>>> * The former causes old requirements to be kept in place, meaning caps,
>>> exclusions, etc. are being kept, which can cause conflicts.
>>>
>>> * Keeping the old version in requirements means that cross dependencies
>>> are not tested with updated versions.
>>>
>>> This has hit us with the mistral and tripleo projects particularly
>>> (tagged in the title).  They disallow pbr-3.0.0 and in the case of
>>> mistral sqlalchemy updates.
>>>
>>> [mistral]
>>> mistral - blocking sqlalchemy - milestones
>>>
>>> [tripleo]
>>> os-refresh-config - blocking pbr - milestones
>>> os-apply-config - blocking pbr - milestones
>>> os-collect-config - blocking pbr - milestones
>>
>> These are cycle-with-milestones., like os-net-config for example,
>> which wasn't mentioned in this email. It has the same releases as
>> os-net-config also, so I'm confused why these 3 cause issue, I
>> probably missed something.
>>
>> Anyway, I'm happy to change os-*-config (from TripleO) to be
>> cycle-with-intermediary. Quick question though, which tag would you
>> like to see, regarding what we already did for pike-1?
>>
>> Thanks,
>>
>
> Pike is fine as it's just master that has this issue.  The problem is
> that the latest release blocks the pbr from upper-constraints from being
> coinstallable.

Done, please review: https://review.openstack.org/#/c/469530/

Thanks,

>>> [nova]
>>> os-vif - blocking pbr - intermediary
>>>
>>> [horizon]
>>> django-openstack-auth - blocking django - intermediary
>>>
>>>
>>> So, here's what needs doing.
>>>
>>> Those projects that are already using the cycle-with-intermediary model
>>> should just do a release.
>>>
>>> For those that are using cycle-with-milestones, you will need to change
>>> to the cycle-with-intermediary model, and do a full release, both can be
>>> done at the same time.
>>>
>>> If anyone has any questions or wants clarifications this thread is good,
>>> or I'm on irc as prometheanfire in the #openstack-requirements channel.
>>>
>>> --
>>> Matthew Thode (prometheanfire)
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>
>
> --
> Matthew Thode (prometheanfire)
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [neutron] tempest failures when deploying neutron-server in wsgi with apache

2017-05-31 Thread Emilien Macchi
Hey folks,

I've been playing with deploying Neutron in WSGI with Apache and
Tempest tests fail on spawning Nova server when creating Neutron
ports:
http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/console.html#_2017-05-30_13_09_22_715400

I haven't found anything useful in neutron-server logs:
http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/logs/apache/neutron_wsgi_access_ssl.txt.gz

Before I file a bug in neutron, can anyone look at the logs with me
and see if I missed something in the config:
http://logs.openstack.org/89/459489/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/f2ee8bf/logs/apache_config/10-neutron_wsgi.conf.txt.gz

Thanks for the help,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements][mistral][tripleo][horizon][nova][releases] release models for projects tracked in global-requirements.txt

2017-05-30 Thread Emilien Macchi
On Tue, May 30, 2017 at 8:36 PM, Matthew Thode
<prometheanf...@gentoo.org> wrote:
> We have a problem in requirements that projects that don't have the
> cycle-with-intermediary release model (most of the cycle-with-milestones
> model) don't get integrated with requirements until the cycle is fully
> done.  This causes a few problems.
>
> * These projects don't produce a consumable release for requirements
> until end of cycle (which does not accept beta releases).
>
> * The former causes old requirements to be kept in place, meaning caps,
> exclusions, etc. are being kept, which can cause conflicts.
>
> * Keeping the old version in requirements means that cross dependencies
> are not tested with updated versions.
>
> This has hit us with the mistral and tripleo projects particularly
> (tagged in the title).  They disallow pbr-3.0.0 and in the case of
> mistral sqlalchemy updates.
>
> [mistral]
> mistral - blocking sqlalchemy - milestones
>
> [tripleo]
> os-refresh-config - blocking pbr - milestones
> os-apply-config - blocking pbr - milestones
> os-collect-config - blocking pbr - milestones

These are cycle-with-milestones., like os-net-config for example,
which wasn't mentioned in this email. It has the same releases as
os-net-config also, so I'm confused why these 3 cause issue, I
probably missed something.

Anyway, I'm happy to change os-*-config (from TripleO) to be
cycle-with-intermediary. Quick question though, which tag would you
like to see, regarding what we already did for pike-1?

Thanks,

> [nova]
> os-vif - blocking pbr - intermediary
>
> [horizon]
> django-openstack-auth - blocking django - intermediary
>
>
> So, here's what needs doing.
>
> Those projects that are already using the cycle-with-intermediary model
> should just do a release.
>
> For those that are using cycle-with-milestones, you will need to change
> to the cycle-with-intermediary model, and do a full release, both can be
> done at the same time.
>
> If anyone has any questions or wants clarifications this thread is good,
> or I'm on irc as prometheanfire in the #openstack-requirements channel.
>
> --
> Matthew Thode (prometheanfire)
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] CI Squad Meeting Summary (week 21) - Devmode OVB, RDO Cloud and config management

2017-05-30 Thread Emilien Macchi
On Fri, May 26, 2017 at 4:58 PM, Attila Darazs <adar...@redhat.com> wrote:
> If the topics below interest you and you want to contribute to the
> discussion, feel free to join the next meeting:
>
> Time: Thursdays, 14:30-15:30 UTC
> Place: https://bluejeans.com/4113567798/
>
> Full minutes: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting
>
> = Periodic & Promotion OVB jobs Quickstart transition =
>
> We had a lively technical discussions this week. Gabriele's work on
> transitioning the periodic & promotion jobs is nearly complete, only needs
> reviews at this point. We won't set a transition date for these as it is not
> really impacting folks long term if these jobs are failing for a few days at
> this point. We'll transition when everything is ready.
>
> = RDO Cloud & Devmode OVB =
>
> We continued planning the introduction of RDO Cloud for the upstream OVB
> jobs. We're still at the point of account setup.
>
> The new OVB based devmode seems to be working fine. If you have access to
> RDO Cloud, and haven't tried it already, give it a go. It can set up a full
> master branch based deployment within 2 hours, including any pending changes
> baked into the under & overcloud.
>
> When you have your account info sourced, all it takes is
>
> $ ./devmode.sh --ovb
>
> from your tripleo-quickstart repo! See here[1] for more info.
>
> = Container jobs on nodepool multinode =
>
> Gabriele is stuck with these new Quickstart jobs. We would need a deep dive
> into debugging and using the container based TripleO deployments. Let us
> know if you can do one!

I've pinged some folks around, let's see if someone volunteers to make it.

> = How to handle Quickstart configuration =
>
> This a never-ending topic, on which we managed to spend a good chunk of time
> this week as well. Where should we put various configs? Should we duplicate
> a bunch of variables or cut them into small files?
>
> For now it seems we can agree on 3 levels of configuration:
>
> * nodes config (i.e. how many nodes we want for the deployment)
> * envionment + provisioner settings (i.e. you want to run on rdocloud with
> ovb, or on a local machine with libvirt)
> * featureset (a certain set of features enabled/disabled for the jobs, like
> pacemaker and ssl)
>
> This seems rather straightforward until we encounter exceptions. We're going
> to figure out the edge cases and rework the current configs to stick to the
> rules.
>
>
> That's it for this week. Thank you for reading the summary.
>
> Best regards,
> Attila
>
> [1] http://docs.openstack.org/developer/tripleo-quickstart/devmode-ovb.html
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Security bug in diskimage-builder

2017-05-30 Thread Emilien Macchi
On Tue, May 30, 2017 at 3:43 PM, Ben Nemec <openst...@nemebean.com> wrote:
>
>
> On 05/30/2017 08:00 AM, Emilien Macchi wrote:
>>
>> On Mon, May 29, 2017 at 9:02 PM, Jeremy Stanley <fu...@yuggoth.org> wrote:
>>>
>>> On 2017-05-29 15:43:43 +0200 (+0200), Emilien Macchi wrote:
>>>>
>>>> On Wed, May 24, 2017 at 7:45 PM, Ben Nemec <openst...@nemebean.com>
>>>> wrote:
>>>
>>> [...]
>>>>>
>>>>> Emilien, I think we should create a tripleo-coresec group in
>>>>> launchpad that can be used for this. We have had
>>>>> tripleo-affecting security bugs in the past and I imagine we
>>>>> will again. I'm happy to help out with that, although I will
>>>>> admit my launchpad-fu is kind of weak so I don't know off the
>>>>> top of my head how to do it.
>>>>
>>>>
>>>> That or re-use an existing Launchpad group used by OpenStack VMT?
>>>
>>>
>>> The OpenStack VMT doesn't triage bugs for deliverables aside from
>>> those tagged with vulnerability:managed in governance. For those we
>>> recommend private security bugs only be automatically shared with
>>> the openstack-vuln-mgmt team in LP, and then we manually subscribe
>>> something-coresec to the report once we're sure it was reported
>>> against the correct project. For deliverables without VMT oversight,
>>> it makes sense to have private security bugs automatically shared
>>> with those something-coresec teams directly.
>>>
>>>
>>> https://governance.openstack.org/tc/reference/tags/vulnerability_managed.html
>>
>>
>> I created https://launchpad.net/~tripleo-coresec
>>
>> With me (Pacific Time soon), shardy (Europe), bnemec (East coast) and
>
>
> If by "coast" you mean the Great Lakes then yes, but I'm in the central time
> zone. ;-)

lol.
I added James to cover (real) East coast, so we cover most of our TZs.

Thanks,

> Thanks for getting this set up guys.
>
>
>> fungi (East coast) for now. If we feel like we need more people we'll
>> think about it.
>> I'll explore Launchpad to see how we can use this group to handle Security
>> bugs.
>>
>> Thanks,
>>
>>> --
>>> Jeremy Stanley
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Security bug in diskimage-builder

2017-05-30 Thread Emilien Macchi
On Tue, May 30, 2017 at 3:10 PM, Jeremy Stanley <fu...@yuggoth.org> wrote:
> On 2017-05-30 15:00:11 +0200 (+0200), Emilien Macchi wrote:
> [...]
>> I'll explore Launchpad to see how we can use this group to handle
>> Security bugs.
>
> I'll save you some time! ;)

Many thanks, indeed it helped.

> Go to https://launchpad.net/tripleo/+sharing (repeat for any other
> projects the TripleO team has on LP) and add a row to that table for
> the new LP team you've created with sharing set to "Private
> Security: All". Also make sure the "Private Security: All" sharing
> option is removed from other teams.
>
> You may also see some rows in that table for individuals or other
> groups who are subscribed to specific private bugs. These show up
> with a sharing setting like "Private Security: Some" and can be
> safely ignored.
>
> Note that access to the sharing settings requires you to be in
> either the Maintainer or Driver group for the project in question (I
> don't remember which).
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Security bug in diskimage-builder

2017-05-30 Thread Emilien Macchi
On Mon, May 29, 2017 at 9:02 PM, Jeremy Stanley <fu...@yuggoth.org> wrote:
> On 2017-05-29 15:43:43 +0200 (+0200), Emilien Macchi wrote:
>> On Wed, May 24, 2017 at 7:45 PM, Ben Nemec <openst...@nemebean.com> wrote:
> [...]
>> > Emilien, I think we should create a tripleo-coresec group in
>> > launchpad that can be used for this. We have had
>> > tripleo-affecting security bugs in the past and I imagine we
>> > will again. I'm happy to help out with that, although I will
>> > admit my launchpad-fu is kind of weak so I don't know off the
>> > top of my head how to do it.
>>
>> That or re-use an existing Launchpad group used by OpenStack VMT?
>
> The OpenStack VMT doesn't triage bugs for deliverables aside from
> those tagged with vulnerability:managed in governance. For those we
> recommend private security bugs only be automatically shared with
> the openstack-vuln-mgmt team in LP, and then we manually subscribe
> something-coresec to the report once we're sure it was reported
> against the correct project. For deliverables without VMT oversight,
> it makes sense to have private security bugs automatically shared
> with those something-coresec teams directly.
>
> https://governance.openstack.org/tc/reference/tags/vulnerability_managed.html

I created https://launchpad.net/~tripleo-coresec

With me (Pacific Time soon), shardy (Europe), bnemec (East coast) and
fungi (East coast) for now. If we feel like we need more people we'll
think about it.
I'll explore Launchpad to see how we can use this group to handle Security bugs.

Thanks,

> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Security bug in diskimage-builder

2017-05-29 Thread Emilien Macchi
On Wed, May 24, 2017 at 7:45 PM, Ben Nemec <openst...@nemebean.com> wrote:
>
>
> On 05/17/2017 10:46 AM, Jeremy Stanley wrote:
>>
>> On 2017-05-17 15:57:16 +0300 (+0300), George Shuklin wrote:
>>>
>>> There is a bug in diskimage-builder I reported it at 2017-03-10 as
>>> 'private
>>> security'. I think this bug is a medium severity.
>>>
>>> So far there was no reaction at all. I plan to change this bug to public
>>> security on next Monday. If someone is interested in bumping up CVE count
>>> for DIB, please look at
>>> https://bugs.launchpad.net/diskimage-builder/+bug/1671842 (private-walled
>>> for security group).
>>
>>
>> Thanks for the heads up! One thing we missed in the migration of DIB
>> from TripleO to Infra team governance is that the bug tracker for it
>> was still under TripleO team control (I just now leveraged my
>> OpenStack Administrator membership on LP to fix that), so the bug
>> was only visible to https://launchpad.net/~tripleo until moments
>> ago.
>>
>> That said, a "private" bug report visible to the 86 people who are
>> members of that LP team doesn't really qualify as private in my book
>> so there's probably no additional harm in just switching it to
>> public security while I work on triaging it with the DIB devs.
>> Going forward, private security bugs filed for DIB are only visible
>> to the 18 people who make up the diskimage-builder-core and
>> openstack-ci-core teams on LP, which is still more than it probably
>> should be but it's a start at least.
>
>
> Hmm, this points out a valid issue that we don't have a security group for
> tripleo at all.  We use the tripleo group to include basically all tripleo
> developers so it's definitely not appropriate for this purpose.
>
> Emilien, I think we should create a tripleo-coresec group in launchpad that
> can be used for this.  We have had tripleo-affecting security bugs in the
> past and I imagine we will again.  I'm happy to help out with that, although
> I will admit my launchpad-fu is kind of weak so I don't know off the top of
> my head how to do it.

That or re-use an existing Launchpad group used by OpenStack VMT?

fungi, thoughts?
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements][tripleo][heat] Projects holding back requirements updates

2017-05-26 Thread Emilien Macchi
On Fri, May 26, 2017 at 11:22 AM, Steven Hardy <sha...@redhat.com> wrote:
> On Thu, May 25, 2017 at 03:15:13PM -0500, Ben Nemec wrote:
>> Tagging with tripleo and heat teams for the os-*-config projects.  I'm not
>> sure which owns them now, but it sounds like we need a new release.
>
> I think they're still owned by the TripleO team, but we missed them in the
> pike-1 release, I pushed this patch aiming to resolve that:

Indeed, we missed it, apologize for that.

> https://review.openstack.org/#/c/468292/

Thanks!

>
> Steve
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] A proposal for hackathon to reduce deploy time of TripleO

2017-05-23 Thread Emilien Macchi
On Tue, May 23, 2017 at 9:40 AM, Emilien Macchi <emil...@redhat.com> wrote:
> On Tue, May 23, 2017 at 6:47 AM, Sagi Shnaidman <sshna...@redhat.com> wrote:
>> Hi, all
>>
>> I'd like to propose an idea to make one or two days hackathon in TripleO
>> project with main goal - to reduce deployment time of TripleO.
>>
>> - How could it be arranged?
>>
>> We can arrange a separate IRC channel and Bluejeans video conference session
>> for hackathon in these days to create a "presence" feeling.
>
> +1 for IRC. We already have #openstack-sprint, that we could re-use.
> Also +1 for video conference, to get face to face interactions,
> promptly and unscheduled.
>
>> - How to participate and contribute?
>>
>> We'll have a few responsibility fields like tripleo-quickstart, containers,
>> storage, HA, baremetal, etc - the exact list should be ready before the
>> hackathon so that everybody could assign to one of these "teams". It's good
>> to have somebody in team to be stakeholder and responsible for organization
>> and tasks.
>
> Before running the sprint, we should first track bugs / blueprints
> related to deployment speed.
> Not everyone in our team understands why some parts of deployments
> take time, so we need to make it visible so everyone can know how they
> can help during the sprint.
>
> Maybe we could create a Launchpad tag "deployment-time" to track bugs
> related to it. We should also make prioritization so we can work on
> the most critical ones first.

Proposed here: https://review.openstack.org/467335
Discussion can happen in Gerrit for this one.

> I like the idea of breaking down the skills into small groups:
>
> - High Availability: deployment & runtime of Pacemaker optimization
> - Puppet: anything related to the steps (a bit more general but only a
> few of us have expertise on it, we could improve it).
> - Heat: work with the Heat team if we have some pending bugs about slowness.
> - Baremetal: ironic / workflows
> - tripleo-quickstart: tasks that can be improved / optimized
>
> This is a proposal ^ feel free to (comment,add|remove) anything.
>
>
>> - What is the goal?
>>
>> The goal of this hackathon to reduce deployment time of TripleO as much as
>> possible.
>>
>> For example part of CI team takes a task to reduce quickstart tasks time. It
>> includes statistics collection, profiling and detection of places to
>> optimize. After this tasks are created, patches are tested and submitted.
>>
>> The prizes will be presented to teams which saved most of time :)
>>
>> What do you think?
>
> Excellent idea, thanks Sagi for proposing it.
>
> Another thought: before doing the sprint, we might want to make sure
> our tripleo-ci is in stable shape (which is not the case right now, we
> have 4 alerts and one of them affects ovb-ha)...
>
>> Thanks
>> --
>> Best regards
>> Sagi Shnaidman
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [release][infra][puppet][stable] Re: [Release-job-failures] Release of openstack/puppet-nova failed

2017-05-23 Thread Emilien Macchi
On Mon, May 22, 2017 at 3:43 PM, Doug Hellmann <d...@doughellmann.com> wrote:
> Excerpts from Jeremy Stanley's message of 2017-05-22 19:16:34 +:
>> On 2017-05-22 12:31:49 -0600 (-0600), Alex Schultz wrote:
>> > On Mon, May 22, 2017 at 10:34 AM, Jeremy Stanley <fu...@yuggoth.org> wrote:
>> > > On 2017-05-22 09:06:26 -0600 (-0600), Alex Schultz wrote:
>> > > [...]
>> > >> We ran into this for the puppet-module-build check job so I created a
>> > >> puppet-agent-install builder.  Perhaps the job needs that added to it
>> > > [...]
>> > >
>> > > Problem here being these repos share the common tarball jobs used
>> > > for generating python sdists, with a little custom logic baked into
>> > > run-tarball.sh[*] for detecting and adjusting when the repo is for a
>> > > Puppet module. I think this highlights the need to create custom
>> > > tarball jobs for Puppet modules, preferably by abstracting this
>> > > custom logic into a new JJB builder.
>> >
>> > I assume you mean a problem if we added this builder to the job
>> > and it fails for some reason thus impacting the python jobs?
>>
>> My concern is more that it increases complexity by further embedding
>> package selection and installation choices into that already complex
>> script. We'd (Infra team) like to get more of the logic out of that
>> random pile of shell scripts and directly into job definitions
>> instead. For one thing, those scripts are only updated when we
>> regenerate our nodepool images (at best once a day) and leads to
>> significant job inconsistencies if we have image upload failures in
>> some providers but not others. In contrast, job configurations are
>> updated nearly instantly (and can even be self-tested in many cases
>> once we're on Zuul v3).
>>
>> > As far as adding to the builder to the job that's not really a
>> > problem and wouldn't change those jobs as they don't reference the
>> > installed puppet executable.
>>
>> It does risk further destabilizing the generic tarball jobs by
>> introducing more outside dependencies which will only be used by a
>> scant handful of the projects running them.
>>
>> > The problem I have with putting this in the .sh is that it becomes
>> > yet another place where we're doing this package installation (we
>> > already do it in puppet openstack in
>> > puppet-openstack-integration). I originally proposed the builder
>> > because it could be reused if a job requires puppet be available.
>> > ie. this case. I'd rather not do what we do in the builder in a
>> > shell script in the job and it seems like this is making it more
>> > complicated than it needs to be when we have to manage this in the
>> > long term.
>>
>> Agreed, I'm saying a builder which installs an unnecessary Puppet
>> toolchain for the generic tarball jobs is not something we'd want,
>> but it would be pretty trivial to make puppet-specific tarball jobs
>> which do use that builder (and has the added benefit that
>> Puppet-specific logic can be moved _out_ of run-tarballs.sh and into
>> your job configuration instead at that point).
>
> That approach makes sense.
>
> When the new job template is set up, let me know so I can add it to the
> release repo validation as a known way to release things.

https://review.openstack.org/467294

Any feedback is welcome,

Thanks!

> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] A proposal for hackathon to reduce deploy time of TripleO

2017-05-23 Thread Emilien Macchi
On Tue, May 23, 2017 at 6:47 AM, Sagi Shnaidman <sshna...@redhat.com> wrote:
> Hi, all
>
> I'd like to propose an idea to make one or two days hackathon in TripleO
> project with main goal - to reduce deployment time of TripleO.
>
> - How could it be arranged?
>
> We can arrange a separate IRC channel and Bluejeans video conference session
> for hackathon in these days to create a "presence" feeling.

+1 for IRC. We already have #openstack-sprint, that we could re-use.
Also +1 for video conference, to get face to face interactions,
promptly and unscheduled.

> - How to participate and contribute?
>
> We'll have a few responsibility fields like tripleo-quickstart, containers,
> storage, HA, baremetal, etc - the exact list should be ready before the
> hackathon so that everybody could assign to one of these "teams". It's good
> to have somebody in team to be stakeholder and responsible for organization
> and tasks.

Before running the sprint, we should first track bugs / blueprints
related to deployment speed.
Not everyone in our team understands why some parts of deployments
take time, so we need to make it visible so everyone can know how they
can help during the sprint.

Maybe we could create a Launchpad tag "deployment-time" to track bugs
related to it. We should also make prioritization so we can work on
the most critical ones first.

I like the idea of breaking down the skills into small groups:

- High Availability: deployment & runtime of Pacemaker optimization
- Puppet: anything related to the steps (a bit more general but only a
few of us have expertise on it, we could improve it).
- Heat: work with the Heat team if we have some pending bugs about slowness.
- Baremetal: ironic / workflows
- tripleo-quickstart: tasks that can be improved / optimized

This is a proposal ^ feel free to (comment,add|remove) anything.


> - What is the goal?
>
> The goal of this hackathon to reduce deployment time of TripleO as much as
> possible.
>
> For example part of CI team takes a task to reduce quickstart tasks time. It
> includes statistics collection, profiling and detection of places to
> optimize. After this tasks are created, patches are tested and submitted.
>
> The prizes will be presented to teams which saved most of time :)
>
> What do you think?

Excellent idea, thanks Sagi for proposing it.

Another thought: before doing the sprint, we might want to make sure
our tripleo-ci is in stable shape (which is not the case right now, we
have 4 alerts and one of them affects ovb-ha)...

> Thanks
> --
> Best regards
> Sagi Shnaidman



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [deployment][tripleo][puppet] using puppet to push configuration into etcd

2017-05-19 Thread Emilien Macchi
While some efforts are made to support k/v to store OpenStack
configurations [1] [2] [3], I spent some time this week to investigate
how Puppet modules could still be used to manage data binding, but
push the config into etcd instead of the files.

A containerized TripleO environment only use Puppet modules to manage
configuration file (and some other bits, but unrelated to $topic). The
configuration management is done in 2 steps: the data binding (exposed
to the end-user) and writing the configuration somewhere (in files
right now e.g. /etc/keystone/keystone.conf). As we are moving to etcd
to store configuration, we still need to maintain a stable interface
for data binding until we figure out the replacement. In other words,
we haven't found yet how we could replace Hiera to bind Heat
parameters into actual OpenStack parameters consumed by the services.

I've been thinking of a transition where we would use etcd to store
configs but we would still use Puppet and Hiera to handle data binding
and push the config into etcd. I think it would make the transition to
etcd smoother since we wouldn't change any logic in parameters and it
would give us more time to find a way to manage the future tool that
will actually push the configuration directly from the interface
(instead of using Puppet). Puppet would still be run during the
TripleO deployment but it wouldn't write any configuration file.
Instead, it would push the config into etcd before or during deploying
containers.

+-+
|TripleO Interface   |
| (Heat Templates, UI, etc) |
+--+
  |
+--v+
| Puppet / Hiera |
++
 |
+--v--+
| etcd |
+--+

I started this WIP work:
https://review.openstack.org/#/c/466292/

It's a first draft of what could be done in puppet-openstacklib to use
etcd as a configuration store backend, instead of configuration file.
During the investigation, I found some limitations to ruby-etcdv3 and
also found out that the openstack_config ruby provider would certainly
require some refacto to be used by etcd backend (a lot of bits are
written for inifile provider).

I'm looking for early feedback on this work, and also potential
contributors willing to help in this effort.

Thanks,

[1] 
https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
[2] https://review.openstack.org/#/c/466109/
[3] https://review.openstack.org/#/c/454897
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] CI Squad Meeting Summary (week 20)

2017-05-19 Thread Emilien Macchi
On Fri, May 19, 2017 at 5:23 AM, Attila Darazs <adar...@redhat.com> wrote:
> If the topics below interest you and you want to contribute to the
> discussion, feel free to join the next meeting:
>
> Time: Thursdays, 14:30-15:30 UTC
> Place: https://bluejeans.com/4113567798/
>
> Full minutes: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting

Sorry I couldn't make it yesterday.

> = Using RDO Cloud for OVB jobs =
>
> We spent some time discussing the steps needed to start running a few OVB
> TripleO jobs on the new RDO Cloud, which seems to be a good shape to start
> utilizing it. We need to create new users for it and add the cloud
> definition to project-config among other things.
>
> When all is set up, we will ramp up the amount of jobs ran there slowly to
> test the stability and bottlenecks.
>
> = Old OVB jobs running without Quickstart =
>
> There are a couple of jobs that is still not transitioned running on a few
> repos. We need to figure out if those jobs are still needed and if yes,
> what's holding them back from transition.
>
> = CI jobs with containers =
>
> We talked about possible ways to update all the containers with fresh and
> gating packages. It's not a trivial problem and we will probably involve
> more container folks in it. The current idea is to create a container that
> could locally serve the DLRN hash packages, avoiding downloading them for
> each containers. However this will be still an IO intensive solution, but
> probably there's no way around it.

Just an FYI about new container jobs: https://review.openstack.org/#/c/466041/
They are now available, feel free to use them with "check experimental".

> = Gate instability, critical bug =
>
> The pingest failures are still plaguing the ovb-ha job, we really need a
> solution for this critical bug[1], as it fails around ~30 percent of the
> time. Please take a look if you can!

The last time I checked on Wednesday, I thought it was a timeout.
Could it be related to the transition to quickstart where some tasks
take more time (like image building, etc).

> Thank you for reading the summary.
>
> Best regards,
> Attila
>
> [1] https://bugs.launchpad.net/tripleo/+bug/1680195
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ptg] How to slice the week to minimize conflicts

2017-05-19 Thread Emilien Macchi
On Thu, May 18, 2017 at 5:27 AM, Thierry Carrez <thie...@openstack.org> wrote:
> Hi everyone,
>
> For the PTG events we have a number of rooms available for 5 days, of
> which we need to make the best usage. We also want to keep it simple and
> productive, so we want to minimize room changes (allocating the same
> room to the same group for one or more days).
>
> For the first PTG in Atlanta, we split the week into two groups.
> Monday-Tuesday for "horizontal" project team meetups (Infra, QA...) and
> workgroups (API WG, Goals helprooms...), and Wednesday-Friday for
> "vertical" project team meetups (Nova, Swift...). This kinda worked, but
> the feedback we received called for more optimizations and reduced
> conflicts.
>
> In particular, some projects which have a lot of contributors overlap
> (Storlets/Swift, or Manila/Cinder) were all considered "vertical" and
> happened at the same time. Also horizontal team members ended up having
> issues to attend workgroups, and had nowhere to go for the rest of the
> week. Finally, on Monday-Tuesday the rooms that had the most success
> were inter-project ones we didn't really anticipate (like the API WG),
> while rooms with horizontal project team meetups were a bit
> under-attended. While we have a lot of constraints, I think we can
> optimize a bit better.
>
> After giving it some thought, my current thinking is that we should
> still split the week in two, but should move away from an arbitrary
> horizontal/vertical split. My strawman proposal would be to split the
> week between inter-project work (+ teams that rely mostly on liaisons in
> other teams) on Monday-Tuesday, and team-specific work on Wednesday-Friday:
>
> Example of Monday-Tuesday rooms:
> Interop WG, Docs, QA, API WG, Packaging WG, Oslo, Goals helproom,
> Infra/RelMgt/support teams helpdesk, TC/SWG room, VM Working group...
>
> Example of Wednesday-Thursday or Wednesday-Friday rooms:
> Nova, Cinder, Neutron, Swift, TripleO, Kolla, Infra...

I like the idea of continuing to have Deployment tools part of
vertical projects room.
Though once it's confirmed, I would like to setup a 2 hours slot where
we meet together and make some cross-deployment-project collaboration.
In Atlanta, we managed to do it on last minute and I found it
extremely useful, let's repeat this but scheduled this time.

> (NB: in this example infra team members end up being available in a
> general support team helpdesk room in the first part of the week, and
> having a regular team meetup on the second part of the week)
>
> In summary, Monday-Tuesday would be mostly around themes, while
> Wednesday-Friday would be mostly around teams. In addition to that,
> teams that /prefer/ to run on Monday-Tuesday to avoid conflicting with
> another project meetup (like Manila wanting to avoid conflicting with
> Cinder, or Storlets wanting to avoid conflicting with Swift) could
> *choose* to go for Monday-Tuesday instead of Wednesday-Friday.
>
> It's a bit of a long shot (we'd still want to equilibrate both sides in
> terms of room usage, so it's likely that the teams that are late to
> decide to participate would be pushed on one side or the other), but I
> think it's a good incremental change that could solve some of the issues
> reported in the Atlanta week slicing, as well as generally make
> inter-project coordination simpler.
>
> If we adopt that format, we need to be pretty flexible in terms of what
> is a "workgroup": to me, any inter-project work that would like to have
> a one-day or two-day room should be able to get some.
> Nova-{Cinder,Neutron,Ironic} discussions would for example happen in the
> VM & BM working group room, but we can imagine others just like it.
>
> Let me know what you think. Also feel free to propose alternate creative
> ways to slice the space and time we'll have. We need to open
> registration very soon (June 1st is the current target), and we'd like
> to have a rough idea of the program before we do that (so that people
> can report which days they will attend more accurately).
>
> --
> Thierry Carrez (ttx)
>
> __________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Future of the tripleo-quickstart-utils project

2017-05-16 Thread Emilien Macchi
Hey Raoul,

Thanks for putting this up in the ML. Replying inline:

On Tue, May 16, 2017 at 4:59 PM, Raoul Scarazzini <ra...@redhat.com> wrote:
> Hi everybody,
> as discussed in today's TripleO meeting [1] here's a brief recap of the
> tripleo-quickstart-utils topic.
>
> ### TL;DR ###
>
> We are trying to understand whether is good or not to put the contents
> of [2] somewhere else for a wider exposure.
>
> ### Long version ###
>
> tripleo-quickstart-utils project started after splitting the
> ha-validation stuff from the tripleo-quickstart-extras repo [3],
> basically because the specificity of the topic was creating a leak of
> reviewers.
> Today this repository have three roles:
>
> 1 - validate-ha: to do ha specific tests depending on the version. This
> role relies on a micro bash framework named ha-test-suite available in
> the same repo, under the utils directory;

I've looked at 
https://github.com/redhat-openstack/tripleo-quickstart-utils/blob/master/roles/validate-ha/tasks/main.yml
and I see it's basically a set of tasks that validates that HA is
working well on the overcloud.
Despite little things that might be adjusted (calling bash scripts
from Ansible), I think this role would be a good fit with
tripleo-validations projects, which is "a collection of Ansible
playbooks to detect and report potential issues during TripleO
deployments".

> 2 - stonith-config: to configure STONITH inside an HA env;

IMHO (and tell me if I'm wrong), this role is something you want to
apply at Day 1 during your deployment, right?
If that's the case, I think the playbooks could really live in THT
where we already have automation to deploy & configure Pacemaker with
Heat and Puppet.
Some tasks might be useful for the upgrade operations but we also have
upgrade_tasks that use Ansible, so possibly easily re-usable.

If it's more Day 2 operations, then we should investigate by creating
a new repository for tripleo with some playbooks useful for Day 2, but
AFIK we've managed to avoid that until now.

> 3 - instance-ha: to configure high availability for instances on the
> compute nodes;

Same as stonith. It sounds like some tasks done during initial
deployment to enable instakce HA and then during upgrade to disable /
enable configurations. I think it could also be done by THT like
stonith configuration.

> Despite of the name, this is not just a tripleo-quickstart related
> project, it is also usable on every TripleO deployed environment, and is
> meant to support all the TripleO OpenStack versions from kilo to pike
> for all the roles it sells;

Great, it means we could easily re-use the bits, modulo some technical
adjustments.

> There's also a docs related to the Multi Virtual Undercloud project [4]
> that explains how to have more than one virtual Undercloud on a physical
> machine to manage more environments from the same place.

I would suggest to move it to tripleo-docs, so we have a single place for doc.

> That's basically the meaning of the word "utils" in the name of the repo.
>
> What I would like to understand is if you see this as something useful
> that can be placed somewhere more near to upstream TripleO project, to
> reach a wider audience for further contribution/evolution.
versus
IIRC, everything in this repo could be moved to existing projects in
TripleO that are already productized, so little efforts would be done.

> ###
>
> [1]
> http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-05-16-14.00.log.html
> [2] https://github.com/redhat-openstack/tripleo-quickstart-utils
> [3]
> https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/validate-ha
> [4]
> https://github.com/redhat-openstack/tripleo-quickstart-utils/tree/master/docs/multi-virtual-undercloud
>
> ###
>
> Thanks for your time,

Thanks for bringing this up!

> --
> Raoul Scarazzini
> ra...@redhat.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tc][infra][release][security][stable][kolla][loci][tripleo][docker][kubernetes] do we want to be publishing binary container images?

2017-05-16 Thread Emilien Macchi
On Tue, May 16, 2017 at 11:12 AM, Doug Hellmann <d...@doughellmann.com> wrote:
> Excerpts from Flavio Percoco's message of 2017-05-16 10:07:52 -0400:
>> On 16/05/17 09:45 -0400, Doug Hellmann wrote:
>> >Excerpts from Flavio Percoco's message of 2017-05-15 21:50:23 -0400:
>> >> On 15/05/17 11:49 -0700, Michał Jastrzębski wrote:
>> >> >On 15 May 2017 at 11:19, Davanum Srinivas <dava...@gmail.com> wrote:
>> >> >> Sorry for the top post, Michal, Can you please clarify a couple of 
>> >> >> things:
>> >> >>
>> >> >> 1) Can folks install just one or two services for their specific 
>> >> >> scenario?
>> >> >
>> >> >Yes, that's more of a kolla-ansible feature and require a little bit
>> >> >of ansible know-how, but entirely possible. Kolla-k8s is built to
>> >> >allow maximum flexibility in that space.
>> >> >
>> >> >> 2) Can the container images from kolla be run on bare docker daemon?
>> >> >
>> >> >Yes, but they need to either override our default CMD (kolla_start) or
>> >> >provide ENVs requred by it, not a huge deal
>> >> >
>> >> >> 3) Can someone take the kolla container images from say dockerhub and
>> >> >> use it without the Kolla framework?
>> >> >
>> >> >Yes, there is no such thing as kolla framework really. Our images
>> >> >follow stable ABI and they can be deployed by any deploy mechanism
>> >> >that will follow it. We have several users who wrote their own deploy
>> >> >mechanism from scratch.
>> >> >
>> >> >Containers are just blobs with binaries in it. Little things that we
>> >> >add are kolla_start script to allow our config file management and
>> >> >some custom startup scripts for things like mariadb to help with
>> >> >bootstrapping, both are entirely optional.
>> >>
>> >> Just as a bonus example, TripleO is currently using kolla images. They 
>> >> used to
>> >> be vanilla and they are not anymore but only because TripleO depends on 
>> >> puppet
>> >> being in the image, which has nothing to do with kolla.
>> >>
>> >> Flavio
>> >>
>> >
>> >When you say "using kolla images," what do you mean? In upstream
>> >CI tests? On contributors' dev/test systems? Production deployments?
>>
>> All of them. Note that TripleO now builds its own "kolla images" (it uses the
>> kolla Dockerfiles and kolla-build) because the dependency of puppet. When I
>> said, TripleO uses kolla images was intended to answer Dims question on 
>> whether
>> these images (or Dockerfiles) can be consumed by other projects.
>>
>> Flavio
>>
>
> Ah, OK. So TripleO is using the build instructions for kolla images, but
> not the binary images being produced today?

Exactly. We have to add Puppet packaging into the list of things we
want in the binary, that's why we don't consume the binary directly.

> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] Project updates - OpenStack Summit

2017-05-16 Thread Emilien Macchi
If you missed the TripleO project updates presentation, feel free to
watch the recording:
https://www.openstack.org/videos/boston-2017/project-update-triple0

and the slides:
https://docs.google.com/presentation/d/1knOesCs3HTqKvIl9iUZciUtE006ff9I3zhxCtbLZz4c

If you have any question or feedback regarding our roadmap, feel free
to use this thread to discuss about it on the public forum.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Validations before upgrades and updates

2017-05-12 Thread Emilien Macchi
On Mon, May 8, 2017 at 7:45 AM, Marios Andreou <mandr...@redhat.com> wrote:
> Hi folks, after some discussion locally with colleagues about improving the
> upgrades experience, one of the items that came up was pre-upgrade and
> update validations. I took an AI to look at the current status of
> tripleo-validations [0] and posted a simple WIP [1] intended to be run
> before an undercloud update/upgrade and which just checks service status. It
> was pointed out by shardy that for such checks it is better to instead
> continue to use the per-service  manifests where possible like [2] for
> example where we check status before N..O major upgrade. There may still be
> some undercloud specific validations that we can land into the
> tripleo-validations repo (thinking about things like the neutron
> networks/ports, validating the current nova nodes state etc?).
>
> So do folks have any thoughts about this subject - for example the kinds of
> things we should be checking - Steve said he had some reviews in progress
> for collecting the overcloud ansible puppet/docker config into an ansible
> playbook that the operator can invoke for upgrade of the 'manual' nodes (for
> example compute in the N..O workflow) - the point being that we can add more
> per-service ansible validation tasks into the service manifests for
> execution when the play is run by the operator - but I'll let Steve point at
> and talk about those.

It looks like a good idea to me. I don't think our operators want to
update / upgrade OpenStack if the cloud is not in a consistent working
state before.

Here's the things we could test:
- Pacemaker cluster health
- Ceph health
- Database
- APIs healthcheck
- RabbitMQ health

> cheers, marios
>
> [0] https://github.com/openstack/tripleo-validations
> [1] https://review.openstack.org/#/c/462918/
> [2]
> https://github.com/openstack/tripleo-heat-templates/blob/stable/ocata/puppet/services/neutron-api.yaml#L197
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] [CI] HA and non-HA OVB jobs are now running with Quickstart

2017-05-12 Thread Emilien Macchi
On Wed, May 10, 2017 at 11:11 AM, Sagi Shnaidman <sshna...@redhat.com> wrote:
> Hi, all
> In addition to multinode jobs, we migrated today part of OVB jobs to use
> quickstart.
>
> We had before OVB ha and OVB nonha jobs and together with migrating them to
> use quickstart we merged them into one job. It's called now:
>  - gate-tripleo-ci-centos-7-ovb-ha-oooq
>
> and will be voting job instead of
>  - gate-tripleo-ci-centos-7-ovb-ha
>  - gate-tripleo-ci-centos-7-ovb-nonha
>
> The updates job "gate-tripleo-ci-centos-7-ovb-updates" stays the same and
> nothing was changed about it. The same is about periodic jobs, they stay the
> same and additional update will be sent when we migrate them too.
>
> In addition for tripleo-ci repository there are two branch jobs:
> - gate-tripleo-ci-centos-7-ovb-ha-oooq-newton
> - gate-tripleo-ci-centos-7-ovb-ha-oooq-ocata
> which replaces accordingly:
>  - gate-tripleo-ci-centos-7-ovb-ha-ocata
>  - gate-tripleo-ci-centos-7-ovb-nonha-ocata
>  - gate-tripleo-ci-centos-7-ovb-ha-newton
>  - gate-tripleo-ci-centos-7-ovb-nonha-newton
>
> A little about "gate-tripleo-ci-centos-7-ovb-ha-oooq" job:
> its features file is located in:
> https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset001.yml
> and it's pretty similar to previous HA job, but in addition it has overcloud
> SSL and nodes introspection enabled (which were tested  in previous non-HA
> job).
>
> The old HA and non-HA jobs are moved into experimental queue and could run
> on the patch with "check experimental". It's done for regression check,
> please use it if you suspect there is a problem with migration.
>
> As usual you are welcome to ask any questions about new jobs and features in
> #tripleo . Tripleo-CI squad folks will be happy to answer you.

Except https://bugs.launchpad.net/tripleo/+bug/1690109 (which doesn't
sound related to quickstart) - I don't see any critical issue until
now.
The runtime looks similar with previous ovb-ha from a quick look, so I
think we're also good here.

Really good work, thanks for everyone involved in this effort!

> Thanks
>
> -- Forwarded message --
> From: Attila Darazs <adar...@redhat.com>
> Date: Wed, Mar 15, 2017 at 12:04 PM
> Subject: [openstack-dev] [tripleo] Gating jobs are now running with
> Quickstart
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
>
>
> As discussed previously in the CI Squad meeting summaries[1] and on the
> TripleO weekly meeting, the multinode gate jobs are now running with
> tripleo-quickstart. To signify the change, we added the -oooq suffix to
> them.
>
> The following jobs migrated yesterday evening, with more to come:
>
> - gate-tripleo-ci-centos-7-undercloud-oooq
> - gate-tripleo-ci-centos-7-nonha-multinode-oooq
> - gate-tripleo-ci-centos-7-scenario001-multinode-oooq
> - gate-tripleo-ci-centos-7-scenario002-multinode-oooq
> - gate-tripleo-ci-centos-7-scenario003-multinode-oooq
> - gate-tripleo-ci-centos-7-scenario004-multinode-oooq
>
> For those who are already familiar with Quickstart, we introduced two new
> concepts:
>
> - featureset config files that are numbered collection of settings, without
> node configuration[2]
> - the '--nodes' option for quickstart.sh and the config/nodes files that
> deal with only the number and type of nodes the deployment will have[3]
>
> If you would like to debug these jobs, it might be useful to read
> Quickstart's documentation[4]. We hope the transition will be smooth, but if
> you have problems ping members of the TripleO CI Squad on #tripleo.
>
> Best regards,
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2017-March/113724.html
> [2]
> https://docs.openstack.org/developer/tripleo-quickstart/feature-configuration.html
> [3]
> https://docs.openstack.org/developer/tripleo-quickstart/node-configuration.html
> [4] https://docs.openstack.org/developer/tripleo-quickstart/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Best regards
> Sagi Shnaidman
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][ci] Upgrade CI job for O->P (containerization)

2017-05-12 Thread Emilien Macchi
On Wed, May 10, 2017 at 9:26 AM, Jiří Stránský <ji...@redhat.com> wrote:
> Hi all,
>
> the upgrade job which tests Ocata -> Pike/master upgrade (from bare-metal to
> containers) just got a green flag from the CI [1].
>
> I've listed the remaining patches we need to land at the very top of the
> container CI etherpad [2], please let's get them reviewed and landed as soon
> as we can. The sooner we get the job going, the fewer upgrade regressions
> will get merged in the meantime (e.g. we have one from last week).
>
> The CI job utilizes mixed release deployment (master undercloud, overcloud
> deployed as Ocata and upgraded to latest). It tests the main overcloud
> upgrade phase (no separate compute role upgrades, no converge phase). This
> means the testing isn't exhaustive to the full expected "production
> scenario", but it covers the most important part where we're likely to see
> the most churn and potential breakages. We'll see how much spare wall time
> we have to add more things once we get the job to run on patches regularly.

The job you and the team made to make that happen is amazing and outstanding.
Once the jobs are considered stable, I would move them to the gate so
we don't break it. Wdyt?

>
> Thanks and have a good day!
>
> Jirka
>
> [1]
> http://logs.openstack.org/61/460061/15/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/d7faa50/
> [2] https://etherpad.openstack.org/p/tripleo-containers-ci
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][ptl][goals] Community goals for Queen

2017-05-05 Thread Emilien Macchi
On Fri, May 5, 2017 at 11:47 AM, Mike Perez <thin...@gmail.com> wrote:
> Hello everyone,
>
> May 11th at 11:00-12:30 at the forum we will be discussing our community wide
> goals for the Queen release [1]!
>
> We do OpenStack-wide goals to "achieve visible common changes, push for basic
> levels of consistency and user experience, and efficiently improve certain
> areas where technical debt payments have become too high – across all 
> OpenStack
> projects" [2].
>
> Our goals backlog: [3]
>
> * New goals are highly welcome.
> * Each goal would achieveable in one cycle, if not we need to break the goal
>   into smaller connected goals.
> * Some Goals already have a team (ex: Python 3) but some haven't.  Maybe could
>   we dress a list of people able to step-up and volunteer to help on these
>   ones.
> * Some Goals might require some documentation for how to achieve it.
>
> Thanks to Emilien for leading our community wide goals for Pike [4]

count on my to be at the session and continue to bring my little contribution.

> * There was an overwhelming amount of support for beginning Python 3.5 support
>   [5] by having our unit tests voting.
> * Unfortunately getting our API endpoints to support WSGI still has some work
>   [6].
>
> Lets start adding/updating what's in our backlog [3] to prepare for the forum
> next week!
>
> [1] - https://governance.openstack.org/tc/goals/pike/index.html
> [2] - https://governance.openstack.org/tc/goals/pike/index.html
> [3] - https://etherpad.openstack.org/p/community-goals
> [4] - https://governance.openstack.org/tc/goals/pike/index.html
>
> --
> Mike Perez
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] pingtest vs tempest

2017-05-04 Thread Emilien Macchi
On Thu, May 4, 2017 at 9:41 AM, Dan Prince <dpri...@redhat.com> wrote:
> On Thu, 2017-05-04 at 03:11 -0400, Luigi Toscano wrote:
>> - Original Message -
>> > On Wed, 2017-05-03 at 17:53 -0400, Emilien Macchi wrote:
>> > > (cross-posting)
>> >
>> > > Instead of running the Pingtest, we would execute a Tempest
>> > > Scenario
>> > > that boot an instance from volume (like Pingstest is already
>> > > doing)
>> > > and see how it goes (in term of coverage and runtime).
>> > > I volunteer to kick-off the work with someone more expert than I
>> > > am
>> > > with quickstart (Arx maybe?).
>> > >
>> > > Another iteration could be to start building an easy interface to
>> > > select which Tempest tests we want a TripleO CI job to run and
>> > > plug
>> > > it
>> > > to our CI tooling (tripleo-quickstart I presume).
>> >
>> > Running a subset of Tempest tests isn't the same thing as designing
>> > (and owning) your own test suite that targets the things that mean
>> > the
>> > most to our community (namely speed and coverage). Even giving up
>> > 5-10
>> > minutes of runtime...just to be able to run Tempest isn't something
>> > that some of us would be willing to do.
>>
>> As I mentioned, you can do it with Tempest (the library). You can
>> have your own test suite that does exactly what you are asking
>> (namely, a set of scenario tests based on Heat which targets the
>> TripleO use case) in a Tempest plugin and there is no absolute reason
>> that those tests should add 5-10 minutes of runtime compared to
>> pingtest.
>>
>> It/they would be exactly pingtest, only implemented using a different
>> library and running with a different runner, with the *exact* same
>> run time.
>>
>> Obvious advantages: only one technology used to run tests, so if
>> anyone else want to run additional tests, there is no need to
>> maintain two code paths; reuse on a big and proven library of test
>> and test runner tools.
>
> I like the idea of getting pingtest out of tripleo.sh as more of a
> stand alone tool. I would support an effort that re-implemented it...
> and using tempest-lib would be totally fine. And as you point out one
> could even combine these tests with a more common "Tempest" run that
> incorporates the scenarios, etc.

I don't understand why we would re-implement the pingtest in a tempest plugin.
Could you please tell us what is the technical difference between what
does this scenario :
https://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py

And this pingtest:
https://github.com/openstack/tripleo-heat-templates/blob/master/ci/pingtests/tenantvm_floatingip.yaml

They both create a volume Cinder, snapshot it in Glance and and spawn
a Nova server from the volume.

What one does that the other one doesn't?

> To me the message is clear that we DO NOT want to consume the normal
> Tempest scenarios in TripleO upstream CI at this point. Sure there is
> overlap there, but the focus of those tests is just plain different...

I haven't seen strong pushback in this thread except you.
I'm against overlap in general and this one is pretty obvious. Why
would we maintain a tripleo-specific Tempest scenario while existing
ones would work for us. Please give me a technical reason in what is
not good enough in the existing scenarios.

> speed isn't a primary concern there as it is for us so I don't think we
> should do it now. And probably not ever unless the CI job time is less
> than an hour. Like even if we were able to tune a set of stock Tempest
> smoke tests today to our liking unless TripleO proper gates on the
> runtime of those not increasing we'd be at risk of breaking our CI
> queues as the wall time would potentially get too long. In this regard
> this entire thread is poorly named I think in that we are no longer
> talking about 'pingtest vs. tempest' but rather the implementation
> details of how we reimplement our existing pingtest to better suite the
> community.

What I would like to see if we're going to use Tempest in our gate, is
to run at least one TripleO jobs as voting in Tempest.

Tempest folks: I need your support here. We have been running Puppet
jobs as non-voting and we have seen a quite number of patches that
broke us because folks were ignore the jobs. If we switch TripleO to
use more Tempest, being in your gate might be required. We'll run the
fastest and  more stable job that we have to make sure the impact for
you is minimal.

> So ++ for the idea of experimenting with the use of tempest.lib. But
> stay away 

Re: [openstack-dev] [tripleo] pingtest vs tempest

2017-05-03 Thread Emilien Macchi
; Lot of install steps actually does not depends on each other,
> it allows more things to be done in parallel, we generally can have more
> core than Ghz.
>
>
>>
>>>
>>>
>>> > Should we focus on how much testing we can get into N time period?
>>> > Then how do we decide an optimal N
>>> > for our constraints?
>>>
>>> Well yeah, but that's pretty much how/why we ended up with pingtest, it's
>>> simple, fast, and provides an efficient way to do smoke tests, e.g
>>> creating
>>> just one heat resource is enough to prove multiple OpenStack services are
>>> running, as well as the DB/RPC etc etc.
>>>
>>> > I've been working on a full up functional test for OpenStack CI builds
>>> > for a long time now, it works but takes
>>> > more than 10 hours. IF you're interested in results kick through to
>>> > Kibana here [0]. Let me know off list if you
>>> > have any issues, the presentation of this data is all experimental
>>> > still.
>>>
>>> This kind of thing is great, and I'd support more exhaustive testing via
>>> periodic jobs etc, but the reality is we need to focus on "bang for buck"
>>> e.g the deepest possible coverage in the most minimal amount of time for
>>> our per-commit tests - we rely on the project gates to provide a full API
>>> surface test, and we need to focus on more basic things like "did the
>>> service
>>> start", and "is the API accessible".  Simple crud operations on a subset
>>> of
>>> the API's is totally fine for this IMO, whether via pingtest or some
>>> other
>>> means.
>>>
>>
>> Right now we do have a periodic job running full tempest, with a few
>> skips, and because of the lack of tempest tests in the patches, it's being
>> pretty hard to keep it stable enough to have a 100% pass, and of course,
>> also the installation very often fails (like in the last five days).
>> For example, [1] is the latest run we have in periodic job that we get
>> results from tempest, and we have 114 failures that was caused by some new
>> code/change, and I have no idea which one was, just looking at the failures,
>> I can notice that smoke tests plus minimum basic scenario tests would catch
>> these failures and the developer could fix it and make me happy :)
>> Now I have to spend several hours installing and debugging each one of
>> those tests to identify where/why it fails.
>> Before this run, we got 100% pass, but unfortunately I don't have the
>> results anymore, it was removed already from logs.openstack.org
>>
>>
>>>
>>> Steve
>>>
>>>
>>> __________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> [1]
>> http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha-tempest-oooq/0072651/logs/oooq/stackviz/#/stdin
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][deployment] Changing to run nova-api/meta-api under uwsgi by default

2017-05-03 Thread Emilien Macchi
On Wed, May 3, 2017 at 9:17 AM, Matt Riedemann <mriede...@gmail.com> wrote:
> Chris Dent has a change up to devstack [1] which is going to make
> nova-api/meta-api run under uwsgi by default on master (Pike).
>
> That depends on a grenade change [2] which is simply there so we can roll
> through upgrades from Ocata to Pike with eventlet, but new installs in Pike
> via devstack will be using uwsgi.
>
> This is just a heads up that the change is happening and once we're doing it
> in devstack then other deployment tooling projects like TripleO and Kolla
> can investigate adding support for running nova-api under uwsgi too.

ack. Thanks for keeping us updated, I think it's very useful.

> [1] https://review.openstack.org/#/c/457715/
> [2] https://review.openstack.org/#/c/460543/
>
> --
>
> Thanks,
>
> Matt
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] no IRC meeting next week

2017-05-02 Thread Emilien Macchi
Hey,

Some of us will be in Boston for the OpenStack Forum, so we don't
handle the weekly IRC meeting next week.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [forum][tripleo] Project Onboarding in Boston

2017-04-29 Thread Emilien Macchi
Hey folks,

If you want to learn more about TripleO or meet the experts, we
organize a TripleO - Project Onboarding on Tue, May 9, 12:45pm –
2:00pm Plaza Level - MR 101.
Sadly, I won't be able to attend it because I have another meeting
that overlaps, and I can't miss it.
But we have a bunch of TripleO developers who will be around.

I kicked-off an etherpad to list folks from TripleO who volunteer to
be there (please add your name) and also to list attendees so we know
how many people we can expect.
https://etherpad.openstack.org/p/BOS-forum-tripleo-onboarding


Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][stable][ptls] Tagging mitaka as EOL

2017-04-28 Thread Emilien Macchi
On Mon, Apr 24, 2017 at 10:43 PM, Emilien Macchi <emil...@redhat.com> wrote:
> On Mon, Apr 17, 2017 at 1:03 PM, Emilien Macchi <emil...@redhat.com> wrote:
>> On Wed, Apr 12, 2017 at 2:47 AM, Tony Breeds <t...@bakeyournoodle.com> wrote:
>>> Hi all,
>>> I'm late in sending this announement, but I'm glad to see several 
>>> projects
>>> have already requested EOL releases to make it trivial and obvious where to
>>> apply the tag.
>>>
>>> I'm proposing to EOL all projects that meet one or more of the following
>>> criteria:
>>>
>>> - The project is openstack-dev/devstack, openstack-dev/grenade or
>>>   openstack/requirements
>>> - The project has the 'check-requirements' job listed as a template in
>>>   project-config:zuul/layout.yaml
>>> - The project gates with either devstack or grenade jobs
>>> - The project is listed in governance:reference/projects.yaml and is tagged
>>>   with 'stable:follows-policy'.
>>>
>>> Some statistics:
>>> All Repos  : 1584 (covered in zuul/layout.yaml)
>>> Checked Repos  :  416 (match one or more of the above 
>>> criteria)
>>> Repos with mitaka branches :  409
>>> EOL Repos  :  199 (repos that match the criteria *and* 
>>> have
>>>a mitaka branch) [1]
>>> NOT EOL Repos  :  210 (repos with a mitaka branch but
>>>otherwise do not match) [2]
>>> DSVM Repos (staying)   :   68 (repos that use dsvm but don't have
>>>mitaka branches)
>>> Tagged Repos   :0
>>> Open Reviews   :  159 (reviews to close)
>>>
>>> Please look over both lists by 2017-04-17 00:00 UTC and let me know if:
>>> - A project is in list 1 and *really* *really* wants to opt *OUT* of EOLing 
>>> and
>>>   why.  Note doing this will amost certainly reduce the testing coverage you
>>>   have in the gate.
>>> - A project is in list 2 that would like to opt *IN* to tagging/EOLing
>>>
>>> Any projects that will be EOL'd will need all open reviews abandoned before 
>>> it
>>> can be processed.  I'm very happy to do this, or if I don't have permissios 
>>> to
>>> do it ask a gerrit admin to do it.
>>
>> Please EOL stable/mitaka for:
>> - instack-undercloud (and featureV2/juno/kilo old branches)
>> - tripleo-heat-templates (and icehouse)
>> - tripleo-puppet-elements
>> - puppet-tripleo
>> - tripleo-image-elements (just featureV2)
>> - python-tripleoclient
>> - tripleo-common
>
> I'll take care of EOLing them, after reading latest email from Tony.
> I would need some help from infra to remove the branches I mentioned ^
> please (featureV2, juno, kilo, icehouse).
>
> I'll keep you posted on my progress.

EOLing for TripleO has been processed.
Big thanks to fungi & Ajaeger for their precious help here.

>> Thanks,
>>
>>> Yours Tony.
>>>
>>> [1] 
>>> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L1
>>> [2] 
>>> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L209
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Emilien Macchi
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] TC Report 17

2017-04-27 Thread Emilien Macchi
is Dent  ┬──┬◡ノ(° -°ノ)   https://anticdent.org/
> freenode: cdent tw: @anticdent
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [puppet] Deprecating Postgresql support

2017-04-27 Thread Emilien Macchi
On Thu, Apr 27, 2017 at 7:38 AM, Emilien Macchi <emil...@redhat.com> wrote:
> Greetings,
>
> We didn't see anyone using Postgresql when deploying Puppet OpenStack:
> - no feedback on ML or IRC
> - no bug report in Launchpad
>
> Postgresql support (or call it how you want) is also removed upstream
> in OpenStack.

I was wrong, the discussion is still ongoing and there is a session
about that in Boston that you might want to participate if you're
deploying PG.

> We will deprecate the class that used to deploy Postgresql in Pike and
> remove the code in Queens.
>
> https://review.openstack.org/#/c/460249/
>
> Any feedback is very welcome before we do it.

We got some feedback from our users who still use PG, so for now,
we're not deprecating anything.

> Thanks,
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack-operators] [puppet] Deprecating Postgresql support

2017-04-27 Thread Emilien Macchi
[adding mailing-lists again]

On Thu, Apr 27, 2017 at 9:31 AM, Benedikt Trefzer <tref...@cirrax.com> wrote:
> Hi Emilien
>
> On 27.04.2017 13:38, Emilien Macchi wrote:
>> Greetings,
>>
>> We didn't see anyone using Postgresql when deploying Puppet OpenStack:
>> - no feedback on ML or IRC
>> - no bug report in Launchpad
>
> Because it just works !
> We use it.

Good to know. That's the kind of information that we like to know
because we're blinding doing stuff for users that we don't know much.

>> Postgresql support (or call it how you want) is also removed upstream
>> in OpenStack.
> Do you have any reference for this ?
> I could not find anything  about this !
>
>> We will deprecate the class that used to deploy Postgresql in Pike and
>> remove the code in Queens.
>>
>> https://review.openstack.org/#/c/460249/
>>
>> Any feedback is very welcome before we do it.
> My Feedback: so sad, but are there any chances to change ?

We hate when our users are sad. No worries, we won't break you.
Right now, the immediate concern is that puppetlabs-postgresql 4.9.0
(latest tag) doesn't work with puppetlabs-concat 4.0.0. We have to pin
it to 3.0.0 in Puppet OpenStack CI and we don't like pins.
If you or someone from your team could fix puppetlabs-postgresql, we
could unpin and be happy again.

On my side, I'm fine to let puppet code that manage postgresql
databases,users,permissions. It's not something 1) breaks often 2)
change often, so the cost of maintenance is imho low. But I'll let the
decision to other folks in our group.

Thanks for this feedback!
Please keep giving more of it, on public channels.

> Greetings
>
> Benedikt
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [puppet] Deprecating Postgresql support

2017-04-27 Thread Emilien Macchi
Greetings,

We didn't see anyone using Postgresql when deploying Puppet OpenStack:
- no feedback on ML or IRC
- no bug report in Launchpad

Postgresql support (or call it how you want) is also removed upstream
in OpenStack.
We will deprecate the class that used to deploy Postgresql in Pike and
remove the code in Queens.

https://review.openstack.org/#/c/460249/

Any feedback is very welcome before we do it.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] Mitaka EOL

2017-04-27 Thread Emilien Macchi
OpenStack Mitaka has been EOLing this month:
https://releases.openstack.org/

We are in the process of EOLing TripleO Mitaka branch.
We did a last release for Mitaka:
https://releases.openstack.org/teams/tripleo.html#mitaka

ovb-ha and ovh-nonha are passing well, so we don't let any critical bug:
https://review.openstack.org/#/c/460223/

All reviews in Gerrit for stable/mitaka have been closed.
This patch will remove all Mitaka jobs in TripleO CI:
https://review.openstack.org/#/c/460347/

Any question / feedback is very welcome,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][stable][ptls] Tagging mitaka as EOL

2017-04-24 Thread Emilien Macchi
On Mon, Apr 17, 2017 at 1:03 PM, Emilien Macchi <emil...@redhat.com> wrote:
> On Wed, Apr 12, 2017 at 2:47 AM, Tony Breeds <t...@bakeyournoodle.com> wrote:
>> Hi all,
>> I'm late in sending this announement, but I'm glad to see several 
>> projects
>> have already requested EOL releases to make it trivial and obvious where to
>> apply the tag.
>>
>> I'm proposing to EOL all projects that meet one or more of the following
>> criteria:
>>
>> - The project is openstack-dev/devstack, openstack-dev/grenade or
>>   openstack/requirements
>> - The project has the 'check-requirements' job listed as a template in
>>   project-config:zuul/layout.yaml
>> - The project gates with either devstack or grenade jobs
>> - The project is listed in governance:reference/projects.yaml and is tagged
>>   with 'stable:follows-policy'.
>>
>> Some statistics:
>> All Repos  : 1584 (covered in zuul/layout.yaml)
>> Checked Repos  :  416 (match one or more of the above 
>> criteria)
>> Repos with mitaka branches :  409
>> EOL Repos  :  199 (repos that match the criteria *and* 
>> have
>>a mitaka branch) [1]
>> NOT EOL Repos  :  210 (repos with a mitaka branch but
>>otherwise do not match) [2]
>> DSVM Repos (staying)   :   68 (repos that use dsvm but don't have
>>mitaka branches)
>> Tagged Repos   :0
>> Open Reviews   :  159 (reviews to close)
>>
>> Please look over both lists by 2017-04-17 00:00 UTC and let me know if:
>> - A project is in list 1 and *really* *really* wants to opt *OUT* of EOLing 
>> and
>>   why.  Note doing this will amost certainly reduce the testing coverage you
>>   have in the gate.
>> - A project is in list 2 that would like to opt *IN* to tagging/EOLing
>>
>> Any projects that will be EOL'd will need all open reviews abandoned before 
>> it
>> can be processed.  I'm very happy to do this, or if I don't have permissios 
>> to
>> do it ask a gerrit admin to do it.
>
> Please EOL stable/mitaka for:
> - instack-undercloud (and featureV2/juno/kilo old branches)
> - tripleo-heat-templates (and icehouse)
> - tripleo-puppet-elements
> - puppet-tripleo
> - tripleo-image-elements (just featureV2)
> - python-tripleoclient
> - tripleo-common

I'll take care of EOLing them, after reading latest email from Tony.
I would need some help from infra to remove the branches I mentioned ^
please (featureV2, juno, kilo, icehouse).

I'll keep you posted on my progress.

> Thanks,
>
>> Yours Tony.
>>
>> [1] 
>> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L1
>> [2] 
>> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L209
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-docs] [tripleo] Creating official Deployment guide for TripleO

2017-04-24 Thread Emilien Macchi
On Fri, Mar 24, 2017 at 11:35 AM, Emilien Macchi <emil...@redhat.com> wrote:
> On Wed, Mar 22, 2017 at 2:25 PM, Emilien Macchi <emil...@redhat.com> wrote:
>> Thanks to these who volunteered, it's appreciated.
>>
>> I'm going to kick-off initial work here:
>>
>> 1. Create CI job that build deploy-guide on tripleo-docs repo (when
>> required) - https://review.openstack.org/448740
>
> Step 1 done.
>
>> 2. In tripleo-docs, modify tox.ini and create initial structure for
>> deployment guide (I'll kick it off once we have the CI job)
>
> Step 2, done and ready for review: https://review.openstack.org/#/c/449684/
>
>> 3. Start moving deployment related bits from doc/source/ to
>> deploy-guide/source/ (I'll need your help for this step). We also need
>> to include Alexendra and her team to make sure we're moving the right
>> bits.
>
> Keep posted, that will be the next step.

So this step is ready now, since we have the directory in place:
https://github.com/openstack/tripleo-docs/tree/master/deploy-guide/source

I would need some help and feedback on the bits that we're going to move:

* what chapters do we want to move? (related to Deployment)
* how do we handle redirection from old URLs (to keep web search valid
until it's indexed again)

>> 4. Expose TripleO deployment guide to deployment guides front page and
>> drink a gin tonic.

I haven't figured yet how to expose the guide to the frontpage but
maybe Alexandra or someone from Docs team can help. It's unclear to me
how to handle that since we have a branchless doc repo, so we would
likely have a versionless documentation link (and handle
version-specific bits inline).


Again, any help is welcome. It would be great to achieve this goal in
Pike: having an official TripleO deployment guide among other
deployment tools guides.

Thanks,

>> I'll give an update when 2. is done so we can start working on the content.
>>
>> Thanks,
>>
>> On Wed, Mar 22, 2017 at 8:22 AM, Flavio Percoco <fla...@redhat.com> wrote:
>>> On 20/03/17 08:01 -0400, Emilien Macchi wrote:
>>>>
>>>> I proposed a blueprint to track the work done:
>>>>
>>>> https://blueprints.launchpad.net/tripleo/+spec/tripleo-deploy-guide
>>>> Target: pike-3
>>>>
>>>> Volunteers to work on it with me, please let me know.
>>>
>>>
>>> It'd be awesome to have some input from the containers squad on this effort
>>> too.
>>> Put me on the list for now while we find another volunteer in the containers
>>> DFG.
>>>
>>> Flavio
>>>
>>>> Thanks,
>>>>
>>>> On Tue, Mar 14, 2017 at 7:00 AM, Alexandra Settle <a.set...@outlook.com>
>>>> wrote:
>>>>>
>>>>> Hey Emilien,
>>>>>
>>>>> You pretty much covered it all! Docs team is happy to provide guidance,
>>>>> but in reality, it should be a fairly straight forward process.
>>>>>
>>>>> The Kolla team just completed their deploy-guide patches and were able to
>>>>> help refine the process a bit further. Hopefully this should help the
>>>>> TripleO team :)
>>>>>
>>>>> Reach out if you have any questions at all :)
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Alex
>>>>>
>>>>> On 3/13/17, 10:32 PM, "Emilien Macchi" <emil...@redhat.com> wrote:
>>>>>
>>>>> Team,
>>>>>
>>>>> [adding Alexandra, OpenStack Docs PTL]
>>>>>
>>>>> It seems like there is a common interest in pushing deployment guides
>>>>> for different OpenStack Deployment projects: OSA, Kolla.
>>>>> The landing page is here:
>>>>> https://docs.openstack.org/project-deploy-guide/newton/
>>>>>
>>>>> And one example:
>>>>>
>>>>> https://docs.openstack.org/project-deploy-guide/openstack-ansible/newton/
>>>>>
>>>>> I think this is pretty awesome and it would bring more visibility for
>>>>> TripleO project, and help our community to find TripleO documentation
>>>>> from a consistent place.
>>>>>
>>>>> The good news, is that openstack-docs team built a pretty solid
>>>>> workflow to make that happen:
>>>>>
>>>>> https://docs.openstack.org/contributor-guide/project-deploy-guide.html
>>>>> An

Re: [openstack-dev] [TripleO][CI] Quickstart deep dive

2017-04-24 Thread Emilien Macchi
On Sun, Apr 23, 2017 at 7:58 PM, Emilien Macchi <emil...@redhat.com> wrote:
> On Sun, Apr 23, 2017 at 3:58 PM, Gabriele Cerami <gcer...@redhat.com> wrote:
>> Hi,
>>
>> with the end of the first phase of the transition of CI to quickstart,
>> it was suggested it could be beneficial to everyone doing a deep dive of
>> quickstart, how to use it, how to contribute to it.
>> I'm proposing next thursday, 27th of april as the date for such a deep
>> dive.
>
> What time?

If it's like usual, it's 1400 UTC.

>> The location: https://bluejeans.com/176756457/
>> The Agenda: https://etherpad.openstack.org/p/quickstart-deep-dive
>> I think we're trying to cover all the needed topic, feel free to add a
>> request on the bottom part of the page.
>>
>> All the informations are stored here
>>
>> https://etherpad.openstack.org/p/tripleo-deep-dive-topics
>>
>> Any feedback welcome.
>
> Thanks, this is really cool.
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Final project logo files

2017-04-24 Thread Emilien Macchi
On Fri, Apr 21, 2017 at 6:06 PM, Heidi Joy Tretheway
<heidi...@openstack.org> wrote:
> Hello TripleO team!
>
> Thanks to your input on the last round (which corrected the owls eyes to have 
> a yellow circle, and make the mascot not cross-eyed), we have a final version 
> for your team.
>
> Here’s a quick look at the logo:
> https://www.dropbox.com/s/ejfa7holov7u1g5/OpenStack_Project_Tripleo_vertical.jpg?dl=0
>
> And here’s a folder with all 10 variations:
> https://www.dropbox.com/sh/zvmw0j1bm14dk6w/AAD2je5lCaB5ZUlLjCjhPlYYa?dl=0
>
> We’ll post these to a public repo and on openstack.org/project-mascots soon. 
> Thanks again for your input!

Thank you very much for this work, unless we have more feedback this
week, you can go ahead and push it online!

>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO][CI] Quickstart deep dive

2017-04-23 Thread Emilien Macchi
On Sun, Apr 23, 2017 at 3:58 PM, Gabriele Cerami <gcer...@redhat.com> wrote:
> Hi,
>
> with the end of the first phase of the transition of CI to quickstart,
> it was suggested it could be beneficial to everyone doing a deep dive of
> quickstart, how to use it, how to contribute to it.
> I'm proposing next thursday, 27th of april as the date for such a deep
> dive.

What time?

> The location: https://bluejeans.com/176756457/
> The Agenda: https://etherpad.openstack.org/p/quickstart-deep-dive
> I think we're trying to cover all the needed topic, feel free to add a
> request on the bottom part of the page.
>
> All the informations are stored here
>
> https://etherpad.openstack.org/p/tripleo-deep-dive-topics
>
> Any feedback welcome.

Thanks, this is really cool.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [deployment][forum][osa][kolla][tripleo][helm] proposing a session about future of configuration management - ops + devs wanted!

2017-04-21 Thread Emilien Macchi
The session will be on Monday and we now have an etherpad:
https://etherpad.openstack.org/p/BOS-forum-future-of-configuration-management
Please start feeding with thoughts and questions!

Again, if you're interested by future of configuration management in
OpenStack, you should really join the discussion.

2.50pm - Room 102

Target: Operators, Deployment tools (TripleO, Kolla, OSA, Helm, etc).


Thanks,

On Tue, Mar 21, 2017 at 6:23 PM, Emilien Macchi <emil...@redhat.com> wrote:
> OpenStack developers and operators who work on deployments: we need you.
>
> http://forumtopics.openstack.org/cfp/details/15
>
> Abstract: I would like to bring Developers and Operators in a room to
> discuss about future of Configuration Management in OpenStack.
>
> Until now, we haven't done a good job in collaborating on how we make
> configuration management in a consistent way across OpenStack
> Deployment Tools.
> Some efforts started to emerge in Pike:
> https://etherpad.openstack.org/p/deployment-pike
> And some projects like TripleO started some discussion on future of
> configuration management:
> https://etherpad.openstack.org/p/tripleo-etcd-transition
>
> In this discussion, we will discuss about our common challenges and
> take some actions from there, where projects could collaborate.
>
> Desired people:
> - Folks from Deployment Tools (TripleO, Kolla, OSA, Kubernetes, etc)
> - Operators who deploy OpenStack
>
> Moderator: me + any volunteer.
>
> Any question on this proposal is very welcome by using this thread.
>
> Thanks for reading so far and I'm looking forward to making progress
> on this topic in Boston.
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] upgrade jobs status

2017-04-20 Thread Emilien Macchi
I found useful to share this super quick status about TripleO Upgrades CI jobs:

# ocata to pike
Upgrade jobs on master have made progress, see results on
https://review.openstack.org/#/c/457603/.
gate-tripleo-ci-centos-7-multinode-upgrades-nv : GREEN
gate-tripleo-ci-centos-7-scenario001-multinode-upgrades-nv: RED - need
investigation
gate-tripleo-ci-centos-7-scenario002-multinode-upgrades-nv: RED -
https://bugs.launchpad.net/tripleo/+bug/1685052
gate-tripleo-ci-centos-7-scenario003-multinode-upgrades-nv: GREEN
gate-tripleo-ci-centos-7-scenario004-multinode-upgrades-nv: GREEN

Once they are all green and not timeouting anymore:
- add them to promotion pipeline
- make them voting

# newton to ocata
Jobs were timeouting and we're attempting to reduce the runtime by
using more AFS mirrors:
https://review.openstack.org/#/c/458474/
I'll give an update by end of Friday or next week to see if it was
really useful.

Any feedback / help is very welcome.
Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [puppet][fuel] stepping down from puppet/fuel core groups

2017-04-20 Thread Emilien Macchi
On Thu, Apr 20, 2017 at 6:44 AM, Denis Egorenko <degore...@mirantis.com> wrote:
> Hello puppet'ers!
>
> Now it's time to say "Goodbay" to you, i'm leaving Mirantis and as well
> stepping down from Puppet/Fuel core groups.
>
> It was a really cool to work with all of you! I've learn a lot! Thank you
> all for all your help and advises.
>
> Also want to especially say "Thank you" to Emilien Macchi, Alex Schultz,
> Sergii Golovatiuk, Vladimir Kuklin and other core guys - you're best! :)

It was a real pleasure to work with you, your impact on the Puppet
modules has been really impressive.
Enjoy your next things and... have fun!

> If you will need my help, please don't hesitate to add me to review or reach
> me: egorenko@gmail.com
>
> Thank you and good luck!
>
> --
> Best Regards,
> Egorenko Denis,
> Senior Deployment Engineer
> Mirantis
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [kolla][all] Core mentorship program kickoff

2017-04-19 Thread Emilien Macchi
On Wed, Apr 19, 2017 at 1:22 PM, Michał Jastrzębski <inc...@gmail.com> wrote:
> Hello everyone,
>
> On todays meeting we officially started mentorship program in Kolla:)
> If you are core or you are interested in becoming one, please sign up
> on this etherpad
>
> https://etherpad.openstack.org/p/kolla-mentorship-signup
>
> Idea is to provide safe environment to ask questions, get feedback
> from trusted person in core team and ultimately join core team.
>
> Role of mentor is:
> 1. Make sure to review changes that your student reviewed, providing
> feedback to his/hers review as well
> 2. Review changes your student proposed
> 3. Answer questions about review process, technical issues and stuff like that
> 4. Be a trusted friend in community:)
> 5. Ultimately, when you decide that your student is ready, feel free
> to kick off voting process for core addition or let me know, I'll do
> it for you
>
> Role of student:
> 1. Review, review, review, your voice counts
> 2. Don't be shy to ask your mentor, either openly or privately
> 3. Care for project en large, care for code and community, it's your
> project and someday you might be mentoring another person:)
>
> I encourage everyone to take part in this program! This is just a
> pilot, we're figuring it out as we go so help us evolve this effort
> and maybe make it more cross-community:)

I just find it very cool and big +1 on cross-community effort if it works.

> Regards,
> Michal
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Proposing Florian Fuchs for tripleo-validations core

2017-04-18 Thread Emilien Macchi
On Tue, Apr 18, 2017 at 6:20 PM, Jason E. Rist <jr...@redhat.com> wrote:
> On 04/18/2017 02:28 AM, Steven Hardy wrote:
>> On Thu, Apr 06, 2017 at 11:53:04AM +0200, Martin André wrote:
>> > Hellooo,
>> >
>> > I'd like to propose we extend Florian Fuchs +2 powers to the
>> > tripleo-validations project. Florian is already core on tripleo-ui
>> > (well, tripleo technically so this means there is no changes to make
>> > to gerrit groups).
>> >
>> > Florian took over many of the stalled patches in tripleo-validations
>> > and is now the principal contributor in the project [1]. He has built
>> > a good expertise over the last months and I think it's time he has
>> > officially the right to approve changes in tripleo-validations.
>> >
>> > Consider this my +1 vote.
>>
>> +1
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> What do we have to do to make this official?

done

> -J
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO][CI] Bridging the production/CI workflow gap with large periodic CI jobs

2017-04-18 Thread Emilien Macchi
On Mon, Apr 17, 2017 at 3:52 PM, Justin Kilpatrick <jkilp...@redhat.com> wrote:
> Because CI jobs tend to max out about 5 nodes there's a whole class of
> minor bugs that make it into releases.
>
> What happens is that they never show up in small clouds, then when
> they do show up in larger testing clouds the people deploying those
> simply work around the issue and get onto what they where supposed to
> be testing. These workarounds do get documented/BZ'd but since they
> don't block anyone and only show up in large environments they become
> hard for developers to fix.
>
> So the issue gets stuck in limbo, with nowhere to test a patchset and
> no one owning the issue.
>
> These issues pile up and pretty soon there is a significant difference
> between the default documented workflow and the 'scale' workflow which
> is filled with workarounds which may or may not be documented
> upstream.
>
> I'd like to propose getting these issues more visibility to having a
> periodic upstream job that uses 20-30 ovb instances to do a larger
> deployment. Maybe at 3am on a Sunday or some other time where there's
> idle execution capability to exploit. The goal being to make these
> sorts of issues more visible and hopefully get better at fixing them.

Wait no, I know some folks at 3am on a Saturday night who use TripleO
CI (ok that was a joke).

> To be honest I'm not sure this is the best solution, but I'm seeing
> this anti pattern across several issues and I think we should try and
> come up with a solution.
>

Yes this proposal is really cool. There is an alternative to run this
periodic scenario outside TripleO CI and send results via email maybe.
But it is something we need to discuss with RDO Cloud people and see
if we would have such resources to make it on a weekly frequency.

Thanks for bringing this up, it's crucial for us to have this kind of
feedback, now let's take actions.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [neutron][tripleo] early feedback about upgrades from ocata to pike

2017-04-18 Thread Emilien Macchi
I've been playing with TripleO uprades from Ocata to Pike and I've
seen that it doesn't work yet. We're currently stuck on Neutron bits:
https://bugs.launchpad.net/tripleo/+bug/1683835

After the upgrade, here are the versions of Neutron:

openstack-neutron-11.0.0-0.20170417080927.b49764c.el7.centos.noarch
(https://github.com/openstack/neutron/commits/b49764c)
openstack-neutron-lbaas-10.0.1-0.20170417085201.071bb03.el7.centos.noarch
(where 071bb03 is the latest commit)
python-neutron-11.0.0-0.20170417080927.b49764c.el7.centos.noarch
python-neutron-lbaas-10.0.1-0.20170417085201.071bb03.el7.centos.noarch
python-neutron-lib-1.3.0-0.20170320164115.cd5b476.el7.centos.noarch
python2-neutronclient-6.1.0-0.20170208193918.1a2820d.el7.centos.noarch

Any help on this one is very welcome,

Thanks!
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tc] [elections] Available time and top priority

2017-04-17 Thread Emilien Macchi
On Mon, Apr 10, 2017 at 4:40 PM, Matt Riedemann <mriede...@gmail.com> wrote:
> On 4/10/2017 2:55 PM, Dean Troyer wrote:
>>
>>
>> The TC meetings are held in IRC and that may somewhat mitigate the
>> issue for non-native English speakers, but I've had problems myself
>> keeping up at times with the flurry of comments.  In any case, I think
>> it would be good to include language in the pile of concerns over
>> world-wide participation
>
>
> I don't attend many TC meetings, it's usually on accident, but yeah, when I
> do I always note the flurry of cross-talk chatter that just drowns
> everything out. I feel like there are usually at least 3 parallel
> conversations going on during a TC meeting and it's pretty frustrating to
> follow along, or get a thought in the mix. That has to be much worse for a
> non-native English speaker.

It is worse and one of the reasons why non-native or not-so-fluent
English speakers have hard time to interact during this meeting.

> So yeah, slow down folks. :)
>
> I'm not advocating splitting the meetings though. It's possible to have your
> cake and eat it to if done properly. For example, Alex Xu runs the Nova API
> subteam meeting and we have people from China, India, Japan, UK and USA and
> get through it fine, but it does involve slowing down to get an
> acknowledgement from people that they are OK with any decisions being made.

I'm personaly not in favor of taking decisions during the TC meeting
for the reasons you just mentioned: timezone & parallel discussions...
I would be in favor of pushing more in async rather than expecting a
lot from this meeting.

> This might also tie back in with what cdent was mentioning, and if the
> flurry of conversation during a TC meeting throws people off, maybe the
> minutes should be digested after the meeting in the mailing list. I know the
> meeting is logged, but it can be hard to read through that without one's
> eyes glazing over due to the cross-talk and locker-room towel whipping going
> on.
>
> --
>
> Thanks,
>
> Matt
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][stable][ptls] Tagging mitaka as EOL

2017-04-17 Thread Emilien Macchi
On Wed, Apr 12, 2017 at 2:47 AM, Tony Breeds <t...@bakeyournoodle.com> wrote:
> Hi all,
> I'm late in sending this announement, but I'm glad to see several projects
> have already requested EOL releases to make it trivial and obvious where to
> apply the tag.
>
> I'm proposing to EOL all projects that meet one or more of the following
> criteria:
>
> - The project is openstack-dev/devstack, openstack-dev/grenade or
>   openstack/requirements
> - The project has the 'check-requirements' job listed as a template in
>   project-config:zuul/layout.yaml
> - The project gates with either devstack or grenade jobs
> - The project is listed in governance:reference/projects.yaml and is tagged
>   with 'stable:follows-policy'.
>
> Some statistics:
> All Repos  : 1584 (covered in zuul/layout.yaml)
> Checked Repos  :  416 (match one or more of the above 
> criteria)
> Repos with mitaka branches :  409
> EOL Repos  :  199 (repos that match the criteria *and* 
> have
>a mitaka branch) [1]
> NOT EOL Repos  :  210 (repos with a mitaka branch but
>otherwise do not match) [2]
> DSVM Repos (staying)   :   68 (repos that use dsvm but don't have
>mitaka branches)
> Tagged Repos   :0
> Open Reviews   :  159 (reviews to close)
>
> Please look over both lists by 2017-04-17 00:00 UTC and let me know if:
> - A project is in list 1 and *really* *really* wants to opt *OUT* of EOLing 
> and
>   why.  Note doing this will amost certainly reduce the testing coverage you
>   have in the gate.
> - A project is in list 2 that would like to opt *IN* to tagging/EOLing
>
> Any projects that will be EOL'd will need all open reviews abandoned before it
> can be processed.  I'm very happy to do this, or if I don't have permissios to
> do it ask a gerrit admin to do it.

Please EOL stable/mitaka for:
- instack-undercloud (and featureV2/juno/kilo old branches)
- tripleo-heat-templates (and icehouse)
- tripleo-puppet-elements
- puppet-tripleo
- tripleo-image-elements (just featureV2)
- python-tripleoclient
- tripleo-common

Thanks,

> Yours Tony.
>
> [1] 
> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L1
> [2] 
> https://gist.github.com/tbreeds/c99e62bf8da19380e4eb130be8783be7#file-mitaka_eol_data-txt-L209
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] pingtest vs tempest

2017-04-17 Thread Emilien Macchi
We haven't got much feedback from TripleO core reviewers, who are
usually more involved on this topic. I'll give a chance to let them
talk because we take some actions based on the feedback brought in
this discussion.

Thanks,

On Fri, Apr 7, 2017 at 4:26 AM, Ravi Sekhar Reddy Konda
<ravisekhar.ko...@oracle.com> wrote:
> Hi Diarmuid,
>
> Even in our setup Sravanthi by mistake has deleted all the ports
> we are trying to bring up again, if it is done today wil ping you and 
> schedule again
>
> Otherwise i will schedule for Monday
>
> Thanks,
> Ravi
> - Original Message -
> From: jtale...@redhat.com
> To: openstack-dev@lists.openstack.org, jkilp...@redhat.com
> Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, 
> New Delhi
> Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest
>
> On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman <sshna...@redhat.com> wrote:
>> HI,
>>
>> I think Rally or Browbeat and other performance oriented solutions won't
>> serve our needs, because we run TripleO CI on virtualized environment with
>> very limited resources. Actually we are pretty close to full utilizing these
>> resources when deploying openstack, so very little is available for test.
>> It's not a problem to run tempest API tests because they are cheap - take
>> little time, little resources, but also gives little coverage. Scenario test
>> are more interesting and gives us more coverage, but also takes a lot of
>> resources (which we don't have sometimes).
>
> Sagi,
> In my original message I mentioned a "targeted" test, I should
> explained that more. We could configure the specific scenario so that
> the load on the virt overcloud would be minimal. Justin Kilpatrick
> already have Browbeat integrated with TripleO Quickstart[1], so there
> shouldn't be much work to try this proposed solution.
>
>>
>> It may be useful to run a "limited edition" of API tests that maximize
>> coverage and don't duplicate, for example just to check service working
>> basically, without covering all its functionality. It will take very little
>> time (i.e. 5 tests for each service) and will give a general picture of
>> deployment success. It will cover fields that are not covered by pingtest as
>> well.
>>
>> I think could be an option to develop a special scenario tempest tests for
>> TripleO which would fit our needs.
>
> I haven't looked at Tempest in a long time, so maybe its functionality
> has improved. I just saw the opportunity to integrate Browbeat/Rally
> into CI to test the functionality of OpenStack services, while also
> capturing performance metrics.
>
> Joe
>
> [1] 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_browbeat_tree_master_ci-2Dscripts=DwIGaQ=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8=Z0jGFw40ezDmSb3F6ns5SXRvacH6AgU0TK5dKBSRgEs=
>
>>
>> Thanks
>>
>>
>> On Wed, Apr 5, 2017 at 11:49 PM, Emilien Macchi <emil...@redhat.com> wrote:
>>>
>>> Greetings dear owls,
>>>
>>> I would like to bring back an old topic: running tempest in the gate.
>>>
>>> == Context
>>>
>>> Right now, TripleO gate is running something called pingtest to
>>> validate that the OpenStack cloud is working. It's an Heat stack, that
>>> deploys a Nova server, some volumes, a glance image, a neutron network
>>> and sometimes a little bit more.
>>> To deploy the pingtest, you obviously need Heat deployed in your
>>> overcloud.
>>>
>>> == Problems:
>>>
>>> Although pingtest has been very helpful over the last years:
>>> - easy to understand, it's an Heat template, like an OpenStack user
>>> would do to deploy their apps.
>>> - fast: the stack takes a few minutes to be created and validated
>>>
>>> It has some limitations:
>>> - Limitation to what Heat resources support (example: some OpenStack
>>> resources can't be managed from Heat)
>>> - Impossible to run a dynamic workflow (test a live migration for example)
>>>
>>> == Solutions
>>>
>>> 1) Switch pingtest to Tempest run on some specific tests, with feature
>>> parity of what we had with pingtest.
>>> For example, we could imagine to run the scenarios that deploys VM and
>>> boot from volume. It would test the same thing as pingtest (details
>>> can be discussed here).
>>> Each scenario would run more tests depending on the service that they
>>&g

[openstack-dev] [all] blog post about being PTL in OpenStack

2017-04-13 Thread Emilien Macchi
Exceptionally, I'll self-promote a blog post that I wrote about my
personal experience of being PTL in OpenStack.

http://my1.fr/blog/my-journey-as-an-openstack-ptl/

My hope is to engage some discussion about what our community thinks
about this role and how we could bring more leaders in OpenStack.
This blog post also explains why I won't run for PTL during the next cycle.

Happy week-end,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][deployment][helm][kolla][openstack-ansible] paunch - a library/tool for yaml driven docker configuration

2017-04-12 Thread Emilien Macchi
[adding more tags to make sure folks can see this thread]

What we would like to hear from Deployment projects (Kolla, OSA, Helm, etc) is:

1) Is there any overlap with some ongoing efforts?
2) Would you be interested by contributing to this effort?

Any feedback is welcome,
Thanks.

On Wed, Apr 12, 2017 at 10:25 PM, Steve Baker <sba...@redhat.com> wrote:
> This is just a heads-up that in a week or so I intend to propose a new git
> repo to be hosted by OpenStack and adopted by the TripleO project.
>
> paunch [1] is a python library and CLI tool which forklifts the logic of the
> docker-cmd heat-agents hook[2] into its own project.
>
> The change to switch the docker-cmd hook to paunch[3] deletes a satisfying
> number of lines of code. Typically a hook is a thin wrapper over another
> configuration tool, and the docker-cmd hook was an unfortunate exception.
>
> The YAML format used by paunch is currently driven by the needs of TripleO
> and is derived from the docker compose v1 format. Over time I'd like to
> evolve the format to faithfully implement defacto standard formats,
> specifically to ease the transition for TripleO to orchestrate containers
> with kubernetes.
>
> At this point I wouldn't advocate for the CLI to be a generally used tool
> for single node container orchestration, but it will gain some commands
> aimed at making developing and deploying containerised TripleO easier.
>
> I'll wait for about a week to get feedback on this proposal, in the meantime
> I'll continue to develop and document the format within [1].
>
> cheers
>
> [1] https://github.com/steveb/paunch
> [2]
> https://github.com/openstack/heat-agents/tree/master/heat-config-docker-cmd
> [3] https://review.openstack.org/#/c/455845/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] pike-1 was released - thank you

2017-04-11 Thread Emilien Macchi
Just a heads-up (while it was mentioned during the TripleO weekly meeting):

We managed to release TripleO pike-1 early this week. This is really
awesome to see the progress we have made and that we continue to do on
the release side. We are continuously improving ourselves and I think
we can be proud of that.
We implemented 8 blueprints and fixed 145 bugs in ~40 days.

I just wanted to thank *you* for this outstanding work and I'm looking
forward to ship the next version of TripleO.

Note: I also want to mention that it wouldn't be possible to achieve
this goal without the external contributors outside TripleO (Infra,
Release management, and other projects in OpenStack). Kudos to them
:-)
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Boston Forum Schedule

2017-04-10 Thread Emilien Macchi
On Mon, Apr 10, 2017 at 7:31 AM, Flavio Percoco <fla...@redhat.com> wrote:
> On 10/04/17 17:35 +0800, Tom Fifield wrote:
>>
>> Hi everyone,
>>
>> Thank you for the many excellent topics submitted for our first Forum. We
>> have updated the topic submission site with the status of each - please
>> check yours.
>>
>> Please also find attached in PDF the proposed schedule for the Forum in
>> Boston.
>>
>>
>> Let us know if you see major issues with it. As Thierry said in design
>> summits past; "It's difficult to make too many changes at this stage as they
>> quickly cascade into breaking all sorts of constraints, but we may still be
>> able to accommodate some." :)
>>
>> Eagle-eyed readers will see that there are a number of slots in gray (on
>> Thursday afternoon). These are being deliberately kept aside for the usual
>> few critical topics that come up in the next few weeks and also for the
>> discoveries we make in the first 3 days of the summit.
>>
>> We'll soon publish the Forum sessions on the main schedule, using the
>> title, abstracts and moderators you submitted, but look forward to your
>> feedback in the mean-time!
>>
>> Thank you all very much for making our first Forum a success.
>>
>>
>
> Hey Folks,
>
> Thanks for working on this.
>
> The topic "Future of configuration management tools" seems to have been
> scheduled twice. Is that expected or a mistake? In the schedule it's not
> explicit whether it's a 2 parts topic or not.

Good point. I think 1 session to keep is enough and I checked both
slots, we can pick one of those, I don't see much conflict that would
prevent folks to have to make an hard choice.

Tom, any chance to remove the second slot please?

Thanks!

> Cheers,
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
> __________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] PTL on leave next week

2017-04-09 Thread Emilien Macchi
I'll be in Ann Arbor for the Leadership Training [1], so I'll be away
from keyboard from Monday afternoon to Thursday night.
I might appear from time to time but don't expect me responsive and
helpful during these days.

Some notes though:

1. I'll handle the TripleO Pike-1 release on my free time, so nothing
to do there.

2.stable/ocata OVB jobs are now broken. I spent little time this
weekend to look but I haven't found much, though you can find my notes
here:
https://bugs.launchpad.net/tripleo/+bug/1680996
It would be great if some folks could have a look this week.

3. I'm not sure that I'll be able to chair the weekly meeting, if
someone could volunteer that would be awesome. It's possible that I
appear during the meeting, but not sure.

If there is anything urgent or any decision that needs to be taken,
please refer to Steve Hardy.

Have a great week everyone!
Thanks,

[1] https://etherpad.openstack.org/p/Leadershiptraining
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo][kolla][openstack-helm][tripleo][all] Storing configuration options in etcd(?)

2017-04-09 Thread Emilien Macchi
On Sun, Apr 9, 2017 at 6:36 PM, Stephen Hindle <shin...@llnw.com> wrote:
> On Fri, Apr 7, 2017 at 3:00 PM, Doug Hellmann <d...@doughellmann.com> wrote:
>>
>> Although I was a proponent of having the reload feature address
>> that issue, I'm not sure the complexity of the current implementation
>> is something we want to hang on to. In the new spec, I propose an
>> alternative treatment, which is to not cache mutable values in the
>> first place so the service never needs to receive a signal to reload.
>> The reload API is retained, and simply discards *everything* and
>> starts the configuration object over as though it was being freshly
>> created. This will be a big change, but the feature is new I think
>> the propose behavior better fits the spirit of the need anyway. Please
>> provide feedback if you think otherwise.
>>
>
> I'm still concerned about how this handles non-OpenStack services.
> Kolla currently provides containers for MySQL, RabbitMQ, Ceph, etc.  I
> agree with Paul Belanger, creating a 'special snowflake' for OpenStack
> config would be bad.  Personally, I agree something like confd could
> be used to keep configs 'out' of the containers, by generating them at
> run time.  This could work for both OS and Non-OS services, giving a
> consistent mechanism.

I still don't see anything wrong that would block us to make OpenStack
services talking to etcd directly, while some non-OpenStack services
wouldn't.
In fact, while oslo.db would get the config from etcd, confd (used by
some non-OpenStack services) would also use etcd backend.

The major thing with confd that I don't like is the fact that it uses
Templates files (like Kolla and some other projects are using). We
don't want to maintain templates files as OpenStack options tend to
change every time.
That's one of the major reasons why we think oslo.db could directly
talk to etcd to store the parameters / values.

>
>
> --
> Stephen Hindle - Senior Systems Engineer
> 480.807.8189 480.807.8189
> www.limelight.com Delivering Faster Better
>
> Join the conversation
>
> at Limelight Connect
>
> --
> The information in this message may be confidential.  It is intended solely
> for
> the addressee(s).  If you are not the intended recipient, any disclosure,
> copying or distribution of the message, or any action or omission taken by
> you
> in reliance on it, is prohibited and may be unlawful.  Please immediately
> contact the sender if you have received this message in error.
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] os-cloud-config retirement

2017-04-07 Thread Emilien Macchi
On Fri, Apr 7, 2017 at 2:27 PM, Andreas Jaeger <a...@suse.com> wrote:
> On 2017-04-07 18:55, Emilien Macchi wrote:
>> os-cloud-config has been retired in Infra and in the repo.
>> RDO packaging has also been updated.
>> Now waiting for final approval in Governance:
>> https://review.openstack.org/#/c/451096/
>>
>> If bug fixes has to happen, please submit to stable branches directly
>> and let us know on #rdo, we'll update the pin.
>
> Emilien, note that the repo is completely read only - on all branches.
> You cannot submit anything anymore,

Ok thanks for the reminder and sorry for the confusion I brought
(TIL). I don't think it's a problem for us.

Thanks!

> Andreas
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] os-cloud-config retirement

2017-04-07 Thread Emilien Macchi
os-cloud-config has been retired in Infra and in the repo.
RDO packaging has also been updated.
Now waiting for final approval in Governance:
https://review.openstack.org/#/c/451096/

If bug fixes has to happen, please submit to stable branches directly
and let us know on #rdo, we'll update the pin.
Thanks,

On Mon, Apr 3, 2017 at 11:56 AM, Emilien Macchi <emil...@redhat.com> wrote:
> The retirement patch is ready whenever you are:
> https://review.openstack.org/#/c/450946/
>
> Please review.
>
> Thanks,
>
> On Mon, Apr 3, 2017 at 3:53 AM, Bogdan Dobrelya <bdobr...@redhat.com> wrote:
>> On 31.03.2017 13:58, Jiří Stránský wrote:
>>> On 30.3.2017 17:39, Juan Antonio Osorio wrote:
>>>> Why not drive the post-config with something like shade over ansible?
>>>> Similar to what the kolla-ansible community is doing.
>>>
>>> We could use those perhaps, if they bring enough benefit to add them to
>>> the container image(s) (i think we'd still want to drive it via a
>>> container rather than fully externally). It's quite tempting to just
>>
>> I hope we still want to decouple configuration from distribution. Wrt
>> versioning issue, custom entry points seem bound to versions of the heat
>> templates and data living there anyway. So it sounds reasonable to me to
>> ship and version entrypoints among templates as a single bundle and
>> please please please keep those out of images.
>>
>>> load a yaml file with the endpoint definitions and just iterate over
>>> them and let Ansible handle the actual API calls...
>>>
>>> However, currently i can't see endpoint management in the cloud modules
>>> docs [1], just service management. Looks like there's still a feature
>>> gap at the moment.
>>>
>>> Jirka
>>
>>
>> --
>> Best regards,
>> Bogdan Dobrelya,
>> Irc #bogdando
>>
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo][kolla][openstack-helm][tripleo][all] Storing configuration options in etcd(?)

2017-04-07 Thread Emilien Macchi
On Thu, Apr 6, 2017 at 7:08 PM, Doug Hellmann <d...@doughellmann.com> wrote:
> Excerpts from Emilien Macchi's message of 2017-04-06 18:17:59 -0400:
>> On Wed, Mar 22, 2017 at 11:23 AM, Flavio Percoco <fla...@redhat.com> wrote:
>> > On 15/03/17 15:40 -0400, Doug Hellmann wrote:
>> >>
>> >> Excerpts from Monty Taylor's message of 2017-03-15 04:36:24 +0100:
>> >>>
>> >>> On 03/14/2017 06:04 PM, Davanum Srinivas wrote:
>> >>> > Team,
>> >>> >
>> >>> > So one more thing popped up again on IRC:
>> >>> > https://etherpad.openstack.org/p/oslo.config_etcd_backend
>> >>> >
>> >>> > What do you think? interested in this work?
>> >>> >
>> >>> > Thanks,
>> >>> > Dims
>> >>> >
>> >>> > PS: Between this thread and the other one about Tooz/DLM and
>> >>> > os-lively, we can probably make a good case to add etcd as a base
>> >>> > always-on service.
>> >>>
>> >>> As I mentioned in the other thread, there was specific and strong
>> >>> anti-etcd sentiment in Tokyo which is why we decided to use an
>> >>> abstraction. I continue to be in favor of us having one known service in
>> >>> this space, but I do think that it's important to revisit that decision
>> >>> fully and in context of the concerns that were raised when we tried to
>> >>> pick one last time.
>> >>>
>> >>> It's worth noting that there is nothing particularly etcd-ish about
>> >>> storing config that couldn't also be done with zk and thus just be an
>> >>> additional api call or two added to Tooz with etcd and zk drivers for it.
>> >>>
>> >>
>> >> The fun* thing about working with these libraries is managing the
>> >> interdependencies. If we're going to have an abstraction library that
>> >> provides configuration options for seeing the backend, like we do in
>> >> oslo.db and olso.messaging, then the configuration library can't use it
>> >> or we have a circular dependency.
>> >>
>> >> Luckily, tooz does not currently use oslo.config. So, oslo.config could
>> >> use tooz and we could create an oslo.dlm library with a shallow
>> >> interface mapping config options to tooz calls to open connections or
>> >> whatever we need from tooz in an application. Then apps could use
>> >> oslo.dlm instead of calling into tooz directly and the configuration of
>> >> the backend would be hidden from the application developer.
>> >
>> >
>> > Replying here becasue I like the proposal, I like what Monty said and I 
>> > also
>> > like what Doug said. Most of the issues and concerns have been covered in
>> > this
>> > thread and I don't have much else to add other than +1.
>>
>> The one-million-dollar question now is: what are the next steps?
>> It sounds like an oslo spec would be nice to summarize the ideas here
>> and talk about design.
>>
>> I volunteer to help but I would need someone more familiar than I am with 
>> Oslo.
>> Please let me know if you're interested to work on it with me
>> otherwise I'll chase chase some of you :-)
>
> I can help from the Oslo side.

I've resurrected an old spec: https://review.openstack.org/#/c/243114/
- addressed some comments and put TODOs that Doug and I will work
together.
The target is set to Queens but we might provide some proof-of-concept
during Pike to make progress.
TripleO project is very interested by this feature and I'm pretty sure
other deployment tools might be too. Any feedback on the work here is
more than welcome!

Thanks,

> Doug
>
>>
>> Thanks for the nice discussions here, I think we've made good progress.
>>
>> >> Doug
>> >>
>> >> * your definition of "fun" may be different than mine
>> >
>> >
>> > Which is probably different than mine :)
>> >
>> > --
>> > @flaper87
>> > Flavio Percoco
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo][kolla][openstack-helm][tripleo][all] Storing configuration options in etcd(?)

2017-04-06 Thread Emilien Macchi
On Wed, Mar 22, 2017 at 11:23 AM, Flavio Percoco <fla...@redhat.com> wrote:
> On 15/03/17 15:40 -0400, Doug Hellmann wrote:
>>
>> Excerpts from Monty Taylor's message of 2017-03-15 04:36:24 +0100:
>>>
>>> On 03/14/2017 06:04 PM, Davanum Srinivas wrote:
>>> > Team,
>>> >
>>> > So one more thing popped up again on IRC:
>>> > https://etherpad.openstack.org/p/oslo.config_etcd_backend
>>> >
>>> > What do you think? interested in this work?
>>> >
>>> > Thanks,
>>> > Dims
>>> >
>>> > PS: Between this thread and the other one about Tooz/DLM and
>>> > os-lively, we can probably make a good case to add etcd as a base
>>> > always-on service.
>>>
>>> As I mentioned in the other thread, there was specific and strong
>>> anti-etcd sentiment in Tokyo which is why we decided to use an
>>> abstraction. I continue to be in favor of us having one known service in
>>> this space, but I do think that it's important to revisit that decision
>>> fully and in context of the concerns that were raised when we tried to
>>> pick one last time.
>>>
>>> It's worth noting that there is nothing particularly etcd-ish about
>>> storing config that couldn't also be done with zk and thus just be an
>>> additional api call or two added to Tooz with etcd and zk drivers for it.
>>>
>>
>> The fun* thing about working with these libraries is managing the
>> interdependencies. If we're going to have an abstraction library that
>> provides configuration options for seeing the backend, like we do in
>> oslo.db and olso.messaging, then the configuration library can't use it
>> or we have a circular dependency.
>>
>> Luckily, tooz does not currently use oslo.config. So, oslo.config could
>> use tooz and we could create an oslo.dlm library with a shallow
>> interface mapping config options to tooz calls to open connections or
>> whatever we need from tooz in an application. Then apps could use
>> oslo.dlm instead of calling into tooz directly and the configuration of
>> the backend would be hidden from the application developer.
>
>
> Replying here becasue I like the proposal, I like what Monty said and I also
> like what Doug said. Most of the issues and concerns have been covered in
> this
> thread and I don't have much else to add other than +1.

The one-million-dollar question now is: what are the next steps?
It sounds like an oslo spec would be nice to summarize the ideas here
and talk about design.

I volunteer to help but I would need someone more familiar than I am with Oslo.
Please let me know if you're interested to work on it with me
otherwise I'll chase chase some of you :-)

Thanks for the nice discussions here, I think we've made good progress.

>> Doug
>>
>> * your definition of "fun" may be different than mine
>
>
> Which is probably different than mine :)
>
> --
> @flaper87
> Flavio Percoco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Proposing Florian Fuchs for tripleo-validations core

2017-04-06 Thread Emilien Macchi
I tripleo-validate that!

OK the joke was poor.

+1 otherwise :-) Nice contributions from Florian, fully deserved.

On Thu, Apr 6, 2017 at 10:18 AM, Jason E. Rist <jr...@redhat.com> wrote:
> On 04/06/2017 07:00 AM, Jiri Tomasek wrote:
>> +1
>>
>> On Thu, Apr 6, 2017 at 12:56 PM, Julie Pichon <jpic...@redhat.com> wrote:
>>
>> > On 6 April 2017 at 10:53, Martin André <m.an...@redhat.com> wrote:
>> >> Hellooo,
>> >>
>> >> I'd like to propose we extend Florian Fuchs +2 powers to the
>> >> tripleo-validations project. Florian is already core on tripleo-ui
>> >> (well, tripleo technically so this means there is no changes to make
>> >> to gerrit groups).
>> >>
>> >> Florian took over many of the stalled patches in tripleo-validations
>> >> and is now the principal contributor in the project [1]. He has built
>> >> a good expertise over the last months and I think it's time he has
>> >> officially the right to approve changes in tripleo-validations.
>> >>
>> >> Consider this my +1 vote.
>> >>
>> >> Martin
>> >>
>> >> [1] http://stackalytics.com/?module=tripleo-validations;
>> > metric=patches=pike
>> >>
>> >
>> > +1!
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> yooge +1
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] pingtest vs tempest

2017-04-05 Thread Emilien Macchi
Greetings dear owls,

I would like to bring back an old topic: running tempest in the gate.

== Context

Right now, TripleO gate is running something called pingtest to
validate that the OpenStack cloud is working. It's an Heat stack, that
deploys a Nova server, some volumes, a glance image, a neutron network
and sometimes a little bit more.
To deploy the pingtest, you obviously need Heat deployed in your overcloud.

== Problems:

Although pingtest has been very helpful over the last years:
- easy to understand, it's an Heat template, like an OpenStack user
would do to deploy their apps.
- fast: the stack takes a few minutes to be created and validated

It has some limitations:
- Limitation to what Heat resources support (example: some OpenStack
resources can't be managed from Heat)
- Impossible to run a dynamic workflow (test a live migration for example)

== Solutions

1) Switch pingtest to Tempest run on some specific tests, with feature
parity of what we had with pingtest.
For example, we could imagine to run the scenarios that deploys VM and
boot from volume. It would test the same thing as pingtest (details
can be discussed here).
Each scenario would run more tests depending on the service that they
run (scenario001 is telemetry, so it would run some tempest tests for
Ceilometer, Aodh, Gnocchi, etc).
We should work at making the tempest run as short as possible, and the
close as possible from what we have with a pingtest.

2) Run custom scripts in TripleO CI tooling, called from the pingtest
(heat template), that would run some validations commands (API calls,
etc).
It has been investigated in the past but never implemented AFIK.

3) ?

I tried to make this text short and go straight to the point, please
bring feedback now. I hope we can make progress on $topic during Pike,
so we can increase our testing coverage and detect deployment issues
sooner.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][TripleO][release][deployment] Packaging problems due to branch/release ordering

2017-04-04 Thread Emilien Macchi
On Tue, Apr 4, 2017 at 9:36 PM, Emilien Macchi <emil...@redhat.com> wrote:
> adding [all] for more visibility... See comments inline:
>
> On Tue, Mar 21, 2017 at 2:02 PM, Emilien Macchi <emil...@redhat.com> wrote:
>> On Mon, Mar 13, 2017 at 12:29 PM, Alan Pevec <ape...@gmail.com> wrote:
>>> 2017-03-09 14:58 GMT+01:00 Jeremy Stanley <fu...@yuggoth.org>:
>>>> In the past we addressed this by automatically merging the release
>>>> tag back into master, but we stopped doing that a cycle ago because
>>>> it complicated release note generation.
>>>
>>> Also this was including RC >= 2 and final tags so as soon as the first
>>> stable maintenance version was released, master was again lower
>>> version.
>>
>> topic sounds staled.
>> Alan,  do we have an ETA on the RDO workaround?
>
> Without progress on RDO tooling and the difficulty of implementing it,
> I went ahead and proposed a semver bump for some projects:
>
> https://review.openstack.org/#/q/topic:sem-ver/pike
>
> Except for Swift where I don't know if they'll bump X, I proposed to bump Y.
> For all other projects, I bumped X as they did from Newton to Ocata.
> (where version is X.Y.Z).
>
> Please give any feedback on the reviews if you prefer another kind of bump.
> Thanks for reviewing that asap, so TripleO CI can test upgrades from
> Ocata to Pike soon.
>
> Thanks,

I forgot to mention that I didn't update Oslo yet, because I have no
clue which type of bump would be the best now.
Please use this url for review:
https://review.openstack.org/#/q/topic:sem-ver/pike+status:open

Thanks,

>> Thanks,
>>
>>> Cheers,
>>> Alan
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> Emilien Macchi
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][TripleO][release][deployment] Packaging problems due to branch/release ordering

2017-04-04 Thread Emilien Macchi
adding [all] for more visibility... See comments inline:

On Tue, Mar 21, 2017 at 2:02 PM, Emilien Macchi <emil...@redhat.com> wrote:
> On Mon, Mar 13, 2017 at 12:29 PM, Alan Pevec <ape...@gmail.com> wrote:
>> 2017-03-09 14:58 GMT+01:00 Jeremy Stanley <fu...@yuggoth.org>:
>>> In the past we addressed this by automatically merging the release
>>> tag back into master, but we stopped doing that a cycle ago because
>>> it complicated release note generation.
>>
>> Also this was including RC >= 2 and final tags so as soon as the first
>> stable maintenance version was released, master was again lower
>> version.
>
> topic sounds staled.
> Alan,  do we have an ETA on the RDO workaround?

Without progress on RDO tooling and the difficulty of implementing it,
I went ahead and proposed a semver bump for some projects:

https://review.openstack.org/#/q/topic:sem-ver/pike

Except for Swift where I don't know if they'll bump X, I proposed to bump Y.
For all other projects, I bumped X as they did from Newton to Ocata.
(where version is X.Y.Z).

Please give any feedback on the reviews if you prefer another kind of bump.
Thanks for reviewing that asap, so TripleO CI can test upgrades from
Ocata to Pike soon.

Thanks,

> Thanks,
>
>> Cheers,
>> Alan
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [puppet] stepping down from puppet-openstack core

2017-04-04 Thread Emilien Macchi
On Tue, Apr 4, 2017 at 5:23 PM, Matt Fischer <tadow...@gmail.com> wrote:
> I am stepping down as core in the puppet openstack project. This is the
> culmination of a long and slow refocus of my work efforts into other areas.
> Additionally I'm not sure what the future holds for me at this point, and
> although it's possible that I will be doing puppet again but it's not fair
> for me to hold this role when I'm not active.
>
> I am very proud of what we, the community, accomplished with these modules
> since I started hacking on them in 2014. The modules went from needing lots
> of work to being very stable and mostly bug free. It took a lot of work and
> some tough decisions from the community to get us to this point and I was
> honored to be a part of it.
>
> Thanks

Thank *you* !

I'll miss the time where you gave strong operator feedback and when we
got issue / solution / patch merged the same day :-)
Anyway, enjoy the next things and thanks for your work here.

PS: I remain available on IRC anytime if you want to continue to
practice your french ;-)

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] Roadmap for Container CI work

2017-04-04 Thread Emilien Macchi
After our weekly meeting of today, I found useful to share and discuss
our roadmap for Container CI jobs in TripleO.
They are ordered by priority from the highest to lowest:

1. Swap ovb-nonha job with ovb-containers, enable introspection on the
container job and shuffle other coverage (e.g ssl) to other jobs
(HA?). It will help us to get coverage for ovb-containers scenario
again, without consuming more rh1 resources and keep existing
coverage.
2. Get multinode coverage of deployments - this should integrate with
the scenarios we already have defined for non-container deployment.
This is super important to cover all overcloud services, like we did
with classic deployments. It should be non voting to start and then
voting once it works. We should find a way to keep the same templates
as we have now, and just include the docker environment. In other
words, find a way to keep using:
https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode.yaml
so we don't duplicate scenario environments.
3. Implement container upgrade job, which for Pike will be deploy a
baremetal overcloud, then migrate on upgrade to containers. Use
multinode jobs for this task. Start with a non-voting job and move to
the gate once it work. I also suggest to use scenarios framework, so
we keep good coverage.
4. After we implement the workflow for minor updates, have a job with
tests container-to-container updates for minor (rolling) updates, this
ideally should add some coverage to ensure no downtime of APIs and
possibly checks for service restarts (ref recent bugs about bouncing
services on minor updates)
5. Once Pike is released and Queens starts, let's work on container to
containers upgrade job.

Any feedback or question is highly welcome,

Note: The proposal comes from shardy's notes on
https://etherpad.openstack.org/p/tripleo-container-ci - feel free to
contribute to the etherpad or mailing list.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [infra] Stop enabling EPEL mirror by default

2017-04-04 Thread Emilien Macchi
On Tue, Apr 4, 2017 at 1:02 PM, Paul Belanger <pabelan...@redhat.com> wrote:
> Greetings,
>
> Recently we've been running into some issues keeping our EPEL mirror properly
> sync'd. We are working to fix this, however we'd also like to do the 
> following:
>
>   Stop enabling EPEL mirror by default
>   https://review.openstack.org/#/c/453222/
>
> For the most part, we enable EPEL for our image build process, this to install
> haveged.  However, it is also likely the majority of centos-7 projects don't
> actually need EPEL.  I know specifically both RDO and TripleO avoid using the
> EPEL repository because of how unstable it is.
>
> Since it is possible this could be a breaking change, jobs will still be able 
> to
> use EPEL, but it will be an opt-in process. Your jobs will need to be updated 
> to
> do:
>
>   $ sudo yum-config-manager --enable epel
>
> Feel free to join us in openstack-infra if you have any questions or concerns.

On behalf of Puppet OpenStack and TripleO groups (both high consumers
of centos-7 nodes): no blocker, go for it.

> -PB
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] CI Squad Meeting Summary (week 13)

2017-04-03 Thread Emilien Macchi
On Mon, Apr 3, 2017 at 8:28 PM, Emilien Macchi <emil...@redhat.com> wrote:
> On Mon, Apr 3, 2017 at 4:41 PM, Paul Belanger <pabelan...@redhat.com> wrote:
>> On Mon, Apr 03, 2017 at 01:08:26PM +0200, Attila Darazs wrote:
>>> If the topics below interest you and you want to contribute to the
>>> discussion, feel free to join the next meeting:
>>>
>>> Time: Thursdays, 14:30-15:30 UTC
>>> Place: https://bluejeans.com/4113567798/
>>>
>>> Full minutes: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting
>>>
>>> We had a meeting full of intense discussion last Thursday. Here's the
>>> summary.
>>>
>>> = Promotion jobs and HTTP caching =
>>>
>>> The first part of it was centered around trying to improve and mostly speed
>>> up the promotion process for TripleO, which is an ongoing discussion for the
>>> last few weeks.
>>>
>>> Image building takes a long time (~30 minutes) for each promotion job, which
>>> we can be spared by having a separate job build the images. This would
>>> result in fewer job timeouts. Zuul v3 will be able to handle these kind of
>>> job dependencies directly, but meanwhile we can probably work around it. Our
>>> contact on this work is pabelanger.
>>>
>>> A lot of other outside queries can be probably sped up by having an
>>> infra-wide caching proxy. This might be an Apache server with mod_proxy for
>>> the short term, and AFS mirror in the long term. This will speed up image
>>> downloads and docker registry downloads as well, speeding up our jobs.
>>>
>> As an update, we did roll out the proxy late last week and so far we setup a
>> reverse proxy cache for https://trunk.rdoproject.org [1]. Currently we just
>> updated puppet jobs to use it, and so far it seems to be working. We are 
>> going
>> to make an announcement to the ML later day for other projects to start using
>> it.
>>
>> We have also started mirroring RDO into AFS, which is our long term solution.
>> This should be ready sometime this week.
>>
>> As for docker things, clarkb and I are planning to test the reverse proxy 
>> with
>> hub.docker.com.  We should know more in a few days, and if the results are 
>> good
>> we'll likely setup a reverse proxy too.
>>
>> Currently, our apache2 configuration is caching files under 100MB up to 2GB 
>> of
>> data. Because we have finite resources on our mirrors, this is the defaults 
>> for
>> today.
>>
>> [1] 
>> http://mirror.regionone.infracloud-vanilla.openstack.org:8080/rdo/centos7/
>
> The only blocker at this time is that rh1 cloud doesn't mirror RDO yet.
> If we want https://review.openstack.org/#/c/451938 to work in the
> gate, we'll have to create it or put some logic to ignore the mirror
> for RDO repo (but I would prefer to have a mirror to optimize the
> deployments).
>
> Thoughts on adding rdo repo to rh1 mirror like Paul did on nodepool providers?

Ok Paul just ran puppet on the mirror and it created the repo:
http://mirror.regionone.tripleo-test-cloud-rh1.openstack.org/rdo/

He asked us to wait a little before using it, so we'll wait his "go".

Thanks Paul,

>>> = Quickstart transition update =
>>>
>>> The big OVB change from last week got merged, now we're checking the
>>> stability of those jobs before proceeding with the transition. We'll want to
>>> have more extensive testing before we move the voting jobs over, so probably
>>> we'll create parallel non-voting jobs this time (ha/non-ha/updates + gate
>>> job), not just testing through pending tripleo-ci changes.
>>>
>>>
>>>
>>> We will probably combine the former ha and nonha OVB jobs to save resources
>>> on rh1. Relevant change and discussion here[1].
>>>
>>> We also briefly talked and discussed how to involve and bring up to speed
>>> more people for reviewing Quickstart changes. There will be a deep dive
>>> session on the subject given by one of the current cores probably.
>>>
>>> Best regards,
>>> Attila
>>>
>>> [1] https://review.openstack.org/449785
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] CI Squad Meeting Summary (week 13)

2017-04-03 Thread Emilien Macchi
On Mon, Apr 3, 2017 at 4:41 PM, Paul Belanger <pabelan...@redhat.com> wrote:
> On Mon, Apr 03, 2017 at 01:08:26PM +0200, Attila Darazs wrote:
>> If the topics below interest you and you want to contribute to the
>> discussion, feel free to join the next meeting:
>>
>> Time: Thursdays, 14:30-15:30 UTC
>> Place: https://bluejeans.com/4113567798/
>>
>> Full minutes: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting
>>
>> We had a meeting full of intense discussion last Thursday. Here's the
>> summary.
>>
>> = Promotion jobs and HTTP caching =
>>
>> The first part of it was centered around trying to improve and mostly speed
>> up the promotion process for TripleO, which is an ongoing discussion for the
>> last few weeks.
>>
>> Image building takes a long time (~30 minutes) for each promotion job, which
>> we can be spared by having a separate job build the images. This would
>> result in fewer job timeouts. Zuul v3 will be able to handle these kind of
>> job dependencies directly, but meanwhile we can probably work around it. Our
>> contact on this work is pabelanger.
>>
>> A lot of other outside queries can be probably sped up by having an
>> infra-wide caching proxy. This might be an Apache server with mod_proxy for
>> the short term, and AFS mirror in the long term. This will speed up image
>> downloads and docker registry downloads as well, speeding up our jobs.
>>
> As an update, we did roll out the proxy late last week and so far we setup a
> reverse proxy cache for https://trunk.rdoproject.org [1]. Currently we just
> updated puppet jobs to use it, and so far it seems to be working. We are going
> to make an announcement to the ML later day for other projects to start using
> it.
>
> We have also started mirroring RDO into AFS, which is our long term solution.
> This should be ready sometime this week.
>
> As for docker things, clarkb and I are planning to test the reverse proxy with
> hub.docker.com.  We should know more in a few days, and if the results are 
> good
> we'll likely setup a reverse proxy too.
>
> Currently, our apache2 configuration is caching files under 100MB up to 2GB of
> data. Because we have finite resources on our mirrors, this is the defaults 
> for
> today.
>
> [1] http://mirror.regionone.infracloud-vanilla.openstack.org:8080/rdo/centos7/

The only blocker at this time is that rh1 cloud doesn't mirror RDO yet.
If we want https://review.openstack.org/#/c/451938 to work in the
gate, we'll have to create it or put some logic to ignore the mirror
for RDO repo (but I would prefer to have a mirror to optimize the
deployments).

Thoughts on adding rdo repo to rh1 mirror like Paul did on nodepool providers?

>> = Quickstart transition update =
>>
>> The big OVB change from last week got merged, now we're checking the
>> stability of those jobs before proceeding with the transition. We'll want to
>> have more extensive testing before we move the voting jobs over, so probably
>> we'll create parallel non-voting jobs this time (ha/non-ha/updates + gate
>> job), not just testing through pending tripleo-ci changes.
>>
>>
>>
>> We will probably combine the former ha and nonha OVB jobs to save resources
>> on rh1. Relevant change and discussion here[1].
>>
>> We also briefly talked and discussed how to involve and bring up to speed
>> more people for reviewing Quickstart changes. There will be a deep dive
>> session on the subject given by one of the current cores probably.
>>
>> Best regards,
>> Attila
>>
>> [1] https://review.openstack.org/449785
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] os-cloud-config retirement

2017-04-03 Thread Emilien Macchi
The retirement patch is ready whenever you are:
https://review.openstack.org/#/c/450946/

Please review.

Thanks,

On Mon, Apr 3, 2017 at 3:53 AM, Bogdan Dobrelya <bdobr...@redhat.com> wrote:
> On 31.03.2017 13:58, Jiří Stránský wrote:
>> On 30.3.2017 17:39, Juan Antonio Osorio wrote:
>>> Why not drive the post-config with something like shade over ansible?
>>> Similar to what the kolla-ansible community is doing.
>>
>> We could use those perhaps, if they bring enough benefit to add them to
>> the container image(s) (i think we'd still want to drive it via a
>> container rather than fully externally). It's quite tempting to just
>
> I hope we still want to decouple configuration from distribution. Wrt
> versioning issue, custom entry points seem bound to versions of the heat
> templates and data living there anyway. So it sounds reasonable to me to
> ship and version entrypoints among templates as a single bundle and
> please please please keep those out of images.
>
>> load a yaml file with the endpoint definitions and just iterate over
>> them and let Ansible handle the actual API calls...
>>
>> However, currently i can't see endpoint management in the cloud modules
>> docs [1], just service management. Looks like there's still a feature
>> gap at the moment.
>>
>> Jirka
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] os-cloud-config retirement

2017-03-30 Thread Emilien Macchi
Hi,

os-cloud-config was deprecated in the Ocata release and is going to be
removed in Pike.

TripleO project doesn't need it anymore and after some investigation
in codesearch.openstack.org, nobody is using it in OpenStack.
I'm working on the removal this cycle, please let us know any concern.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] container jobs are unstable

2017-03-29 Thread Emilien Macchi
On Mon, Mar 27, 2017 at 8:00 AM, Flavio Percoco <fla...@redhat.com> wrote:
> On 23/03/17 16:24 +0100, Martin André wrote:
>>
>> On Wed, Mar 22, 2017 at 2:20 PM, Dan Prince <dpri...@redhat.com> wrote:
>>>
>>> On Wed, 2017-03-22 at 13:35 +0100, Flavio Percoco wrote:
>>>>
>>>> On 22/03/17 13:32 +0100, Flavio Percoco wrote:
>>>> > On 21/03/17 23:15 -0400, Emilien Macchi wrote:
>>>> > > Hey,
>>>> > >
>>>> > > I've noticed that container jobs look pretty unstable lately; to
>>>> > > me,
>>>> > > it sounds like a timeout:
>>>> > > http://logs.openstack.org/19/447319/2/check-tripleo/gate-tripleo-
>>>> > > ci-centos-7-ovb-containers-oooq-nv/bca496a/console.html#_2017-03-
>>>> > > 22_00_08_55_358973
>>>> >
>>>> > There are different hypothesis on what is going on here. Some
>>>> > patches have
>>>> > landed to improve the write performance on containers by using
>>>> > hostpath mounts
>>>> > but we think the real slowness is coming from the images download.
>>>> >
>>>> > This said, this is still under investigation and the containers
>>>> > squad will
>>>> > report back as soon as there are new findings.
>>>>
>>>> Also, to be more precise, Martin André is looking into this. He also
>>>> fixed the
>>>> gate in the last 2 weeks.
>>>
>>>
>>> I spoke w/ Martin on IRC. He seems to think this is the cause of some
>>> of the failures:
>>>
>>> http://logs.openstack.org/32/446432/1/check-tripleo/gate-tripleo-ci-cen
>>> tos-7-ovb-containers-oooq-nv/543bc80/logs/oooq/overcloud-controller-
>>> 0/var/log/extra/docker/containers/heat_engine/log/heat/heat-
>>> engine.log.txt.gz#_2017-03-21_20_26_29_697
>>>
>>>
>>> Looks like Heat isn't able to create Nova instances in the overcloud
>>> due to "Host 'overcloud-novacompute-0' is not mapped to any cell'. This
>>> means our cells initialization code for containers may not be quite
>>> right... or there is a race somewhere.
>>
>>
>> Here are some findings. I've looked at time measures from CI for
>> https://review.openstack.org/#/c/448533/ which provided the most
>> recent results:
>>
>> * gate-tripleo-ci-centos-7-ovb-ha [1]
>>undercloud install: 23
>>overcloud deploy: 72
>>total time: 125
>> * gate-tripleo-ci-centos-7-ovb-nonha [2]
>>undercloud install: 25
>>overcloud deploy: 48
>>total time: 122
>> * gate-tripleo-ci-centos-7-ovb-updates [3]
>>undercloud install: 24
>>overcloud deploy: 57
>>total time: 152
>> * gate-tripleo-ci-centos-7-ovb-containers-oooq-nv [4]
>>undercloud install: 28
>>overcloud deploy: 48
>>total time: 165 (timeout)
>>
>> Looking at the undercloud & overcloud install times, the most task
>> consuming tasks, the containers job isn't doing that bad compared to
>> other OVB jobs. But looking closer I could see that:
>> - the containers job pulls docker images from dockerhub, this process
>> takes roughly 18 min.
>
>
> I think we can optimize this a bit by having the script that populates the
> local
> registry in the overcloud job to run in parallel. The docker daemon can do
> multiple pulls w/o problems.
>
>> - the overcloud validate task takes 10 min more than it should because
>> of the bug Dan mentioned (a fix is in the queue at
>> https://review.openstack.org/#/c/448575/)
>
>
> +A
>
>> - the postci takes a long time with quickstart, 13 min (4 min alone
>> spent on docker log collection) whereas it takes only 3 min when using
>> tripleo.sh
>
>
> mmh, does this have anything to do with ansible being in between? Or is that
> time specifically for the part that gets the logs?
>
>>
>> Adding all these numbers, we're at about 40 min of additional time for
>> oooq containers job which is enough to cross the CI job limit.
>>
>> There is certainly a lot of room for optimization here and there and
>> I'll explore how we can speed up the containers CI job over the next
>
>
> Thanks a lot for the update. The time break down is fantastic,
> Flavio

TBH the problem is far from being solved:

1. Click on https://status-tripleoci.rhcloud.com/
2. Select gate-tripleo-ci-centos-7-ovb-containers-oooq-nv

Container job has been failing more than 55% of the time.

As a referen

Re: [openstack-dev] All patches failing CI for validations stable/ocata branch - help needed

2017-03-29 Thread Emilien Macchi
On Wed, Mar 29, 2017 at 12:32 PM, Florian Fuchs <flfu...@redhat.com> wrote:
> Hi all,
>
> currently all branches proposed for the newly created stable/ocata
> branch on tripleo-validations are failing CI, and the failures seem to
> be unrelated to the content of the patches.[1]
>
> Interestingly, the 3 gates seem to succeed/fail randomly (after a few
> rechecks, some gates failed that succeeded before and vice versa). The
> error in the logs however is always the same [2] and appears to be
> related to an issue with setuptools 34[3], which apparently hit other
> openstack projects as well. I tried to create a patch with a fix
> (excluding setuptools 34), but couldn't test it because gerrit set the
> target patch to master instead of stable/ocata (I guess it only allows
> me to propose backport patches to stable branches, not creating a
> patch for a stable branch only).
>
> At this point I'm a little out of ideas and would appreciate any help
> I can get from folks who are a little more savvy with CI.

In the meantime, I suggest we pin setuptools (short term solution, so
we can continue the dev process on this repo).
https://github.com/openstack/openstack-ansible/commit/335a0c8ba1340ac2a1e88e78ae18f426a0447dcc

> Thanks!
> Florian
>
>
> [1] 
> https://review.openstack.org/#/q/status:open+project:openstack/tripleo-validations+branch:stable/ocata
> [2] IOError: No such file or directory:
> '/home/jenkins/workspace/[...]/pyparsing-2.2.0.dist-info/METADATA'
> [3] https://github.com/pypa/setuptools/issues/951
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [puppet] Stepping down from puppet core

2017-03-29 Thread Emilien Macchi
On Wed, Mar 29, 2017 at 10:57 AM, Ivan Berezovskiy
<iberezovs...@mirantis.com> wrote:
> Hello Puppeteers!
>
> I'd like to step down as a core review for the OpenStack Puppet.
> Unfortunately, as you could see I didn't participate in Puppet
> activities last few months. My career objective has changed and now
> I won't be able even to take a look on OpenStack Puppet.
>
> This was an exciting experience for me to work with you and I will never
> forget our meetings in Austin. We did several releases together, we
> applied common patterns for all our modules, implemented clear integration
> process for new projects and many many other cool stuff.
> I am glad that I worked with such strong leaders as Alex and Emilien!
>
> Puppeteers, I will miss all of you!

http://gph.is/1FIMZ4n

Ivan, thanks for your hard work and I hope you're having fun in your
new position.
Door is always open for you if you decide to come back.

Thanks again!

>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] spec-lite process for tripleo

2017-03-28 Thread Emilien Macchi
Bringing an old topic on the table.

We might have noticed:

1. Some tripleo-specs take huge amount of time before getting merged
(or even reviewed). We have been asking folks to review them every
week but unfortunately they don't get much attraction (# of core
reviewers versus # of folks actually reviewing specs).
2. Some folks spend a lot of time writing TripleO specs and wait for
feedback before starting some implementation (like proof of concept).

Because TripleO like innovation and also moving fast, I think it's
time to bring the tripleo-specs topic on the table:

1. If you have an idea, don't feel obliged to write a specs. Create a
blueprint on launchpad, announce it on the ML and start writing code
(can be real implementation or just a simple PoC). Feedback will be
given in the classic code review.
2. If you still want to write a spec, please make it readable and
communicate about it. If your spec is 900 lines long and not announced
anywhere, there is an high change that it will never been reviewed.
3. If you still want to write a spec, please don't stop your work
after the specs. Please engage some PoC and prototypes to get feedback
directly on the implementation.

I think these proposals are realistic with the regard of how TripleO
project works now.
Please bring any feedback or question.

Thanks,

On Fri, Jan 29, 2016 at 3:26 AM, Dougal Matthews <dou...@redhat.com> wrote:
>
>
> On 27 January 2016 at 16:21, Derek Higgins <der...@redhat.com> wrote:
>>
>> Hi All,
>>
>> We briefly discussed feature tracking in this weeks tripleo meeting. I
>> would like to provide a way for downstream consumers (and ourselves) to
>> track new features as they get implemented. The main things that came out of
>> the discussion is that people liked the spec-lite process that the glance
>> team are using.
>>
>> I'm proposing we would start to use the same process, essentially small
>> features that don't warrant a blueprint would instead have a wishlist bug
>> opened against them and get marked with the spec-lite tag. This bug could
>> then be referenced in the commit messages. For larger features blueprints
>> can still be used. I think the process documented by glance[1] is a good
>> model to follow so go read that and see what you think
>>
>> The general feeling at the meeting was +1 to doing this[2] so I hope we
>> can soon start enforcing it, assuming people are still happy to proceed?
>
>
> +1 sounds good.
>
>>
>>
>> thanks,
>> Derek.
>>
>> [1]
>> http://docs.openstack.org/developer/glance/contributing/blueprints.html#glance-spec-lite
>> [2]
>> http://eavesdrop.openstack.org/meetings/tripleo/2016/tripleo.2016-01-26-14.02.log.html
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Tripleo] Status of monitoring tools

2017-03-28 Thread Emilien Macchi
On Tue, Mar 28, 2017 at 7:04 AM, Sanjay Upadhyay <supad...@redhat.com> wrote:
> Hi Folks,
>
> I recently started work with a requirement to integrate a security tool with
> monitoring framework for alerting and logging. I fail to  find relevant docs
> in this direction. After looking around for more information, it seems we do
> not have the server side implementation at all.
>
> I have created a bug for this
> https://bugs.launchpad.net/tripleo/+bug/1676407. IMO, architecturally we
> have no place for opstools in the tripleo deployment, yet.

I see a perfect fit for using the future TripleO Deployment guide that
we are starting:
https://review.openstack.org/#/c/449684/

We're going to move the content from tripleo-docs/doc/source/ (only
bits for deployments), so you could start from there and push the bits
in the deploy/ foler.

What do you think?

> Ideally, there could be a role like MonServer, which can be a specific role
> for all the monitoring services (server side). If one wants to reduce the no
> of nodes can probably have all the monitoring services on the controller
> node.

Wait, the monitoring server isn't managed by TripleO right? So you
want to document how to deploy OpsTools applications (server side) in
TripleO Doc?
Please correct me if I'm wrong.

> I am still looking at directions on monitoring services.
> regards
> /sanjay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] Propose Attila Darazs and Gabriele Cerami for tripleo-ci core

2017-03-28 Thread Emilien Macchi
I went ahead and added you to tripleo-core Gerrit group.
Keep in mind we expect you to only use the +2 on
openstack-infra/tripleo-ci repository for now.

Thanks for your hard work!

On Tue, Mar 21, 2017 at 12:23 PM, Julie Pichon <jpic...@redhat.com> wrote:
> On 15 March 2017 at 15:44, John Trowbridge <tr...@redhat.com> wrote:
>> Both Attila and Gabriele have been rockstars with the work to transition
>> tripleo-ci to run via quickstart, and both have become extremely
>> knowledgeable about how tripleo-ci works during that process. They are
>> both very capable of providing thorough and thoughtful reviews of
>> tripleo-ci patches.
>>
>> On top of this Attila has greatly increased the communication from the
>> tripleo-ci squad as the liason, with weekly summary emails of our
>> meetings to this list.
>
> +1
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-docs] [tripleo] Creating official Deployment guide for TripleO

2017-03-24 Thread Emilien Macchi
On Wed, Mar 22, 2017 at 2:25 PM, Emilien Macchi <emil...@redhat.com> wrote:
> Thanks to these who volunteered, it's appreciated.
>
> I'm going to kick-off initial work here:
>
> 1. Create CI job that build deploy-guide on tripleo-docs repo (when
> required) - https://review.openstack.org/448740

Step 1 done.

> 2. In tripleo-docs, modify tox.ini and create initial structure for
> deployment guide (I'll kick it off once we have the CI job)

Step 2, done and ready for review: https://review.openstack.org/#/c/449684/

> 3. Start moving deployment related bits from doc/source/ to
> deploy-guide/source/ (I'll need your help for this step). We also need
> to include Alexendra and her team to make sure we're moving the right
> bits.

Keep posted, that will be the next step.

> 4. Expose TripleO deployment guide to deployment guides front page and
> drink a gin tonic.
>
> I'll give an update when 2. is done so we can start working on the content.
>
> Thanks,
>
> On Wed, Mar 22, 2017 at 8:22 AM, Flavio Percoco <fla...@redhat.com> wrote:
>> On 20/03/17 08:01 -0400, Emilien Macchi wrote:
>>>
>>> I proposed a blueprint to track the work done:
>>>
>>> https://blueprints.launchpad.net/tripleo/+spec/tripleo-deploy-guide
>>> Target: pike-3
>>>
>>> Volunteers to work on it with me, please let me know.
>>
>>
>> It'd be awesome to have some input from the containers squad on this effort
>> too.
>> Put me on the list for now while we find another volunteer in the containers
>> DFG.
>>
>> Flavio
>>
>>> Thanks,
>>>
>>> On Tue, Mar 14, 2017 at 7:00 AM, Alexandra Settle <a.set...@outlook.com>
>>> wrote:
>>>>
>>>> Hey Emilien,
>>>>
>>>> You pretty much covered it all! Docs team is happy to provide guidance,
>>>> but in reality, it should be a fairly straight forward process.
>>>>
>>>> The Kolla team just completed their deploy-guide patches and were able to
>>>> help refine the process a bit further. Hopefully this should help the
>>>> TripleO team :)
>>>>
>>>> Reach out if you have any questions at all :)
>>>>
>>>> Thanks,
>>>>
>>>> Alex
>>>>
>>>> On 3/13/17, 10:32 PM, "Emilien Macchi" <emil...@redhat.com> wrote:
>>>>
>>>> Team,
>>>>
>>>> [adding Alexandra, OpenStack Docs PTL]
>>>>
>>>> It seems like there is a common interest in pushing deployment guides
>>>> for different OpenStack Deployment projects: OSA, Kolla.
>>>> The landing page is here:
>>>> https://docs.openstack.org/project-deploy-guide/newton/
>>>>
>>>> And one example:
>>>>
>>>> https://docs.openstack.org/project-deploy-guide/openstack-ansible/newton/
>>>>
>>>> I think this is pretty awesome and it would bring more visibility for
>>>> TripleO project, and help our community to find TripleO documentation
>>>> from a consistent place.
>>>>
>>>> The good news, is that openstack-docs team built a pretty solid
>>>> workflow to make that happen:
>>>>
>>>> https://docs.openstack.org/contributor-guide/project-deploy-guide.html
>>>> And we don't need to create new repos or do any crazy changes. It
>>>> would probably be some refactoring and sphinx things.
>>>>
>>>> Alexandra, please add any words if I missed something obvious.
>>>>
>>>> Feedback from the team would be welcome here before we engage any
>>>> work,
>>>>
>>>> Thanks!
>>>> --
>>>> Emilien Macchi
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Emilien Macchi
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> --
>> @flaper87
>> Flavio Percoco
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-docs] [tripleo] Creating official Deployment guide for TripleO

2017-03-22 Thread Emilien Macchi
Thanks to these who volunteered, it's appreciated.

I'm going to kick-off initial work here:

1. Create CI job that build deploy-guide on tripleo-docs repo (when
required) - https://review.openstack.org/448740
2. In tripleo-docs, modify tox.ini and create initial structure for
deployment guide (I'll kick it off once we have the CI job)
3. Start moving deployment related bits from doc/source/ to
deploy-guide/source/ (I'll need your help for this step). We also need
to include Alexendra and her team to make sure we're moving the right
bits.
4. Expose TripleO deployment guide to deployment guides front page and
drink a gin tonic.

I'll give an update when 2. is done so we can start working on the content.

Thanks,

On Wed, Mar 22, 2017 at 8:22 AM, Flavio Percoco <fla...@redhat.com> wrote:
> On 20/03/17 08:01 -0400, Emilien Macchi wrote:
>>
>> I proposed a blueprint to track the work done:
>>
>> https://blueprints.launchpad.net/tripleo/+spec/tripleo-deploy-guide
>> Target: pike-3
>>
>> Volunteers to work on it with me, please let me know.
>
>
> It'd be awesome to have some input from the containers squad on this effort
> too.
> Put me on the list for now while we find another volunteer in the containers
> DFG.
>
> Flavio
>
>> Thanks,
>>
>> On Tue, Mar 14, 2017 at 7:00 AM, Alexandra Settle <a.set...@outlook.com>
>> wrote:
>>>
>>> Hey Emilien,
>>>
>>> You pretty much covered it all! Docs team is happy to provide guidance,
>>> but in reality, it should be a fairly straight forward process.
>>>
>>> The Kolla team just completed their deploy-guide patches and were able to
>>> help refine the process a bit further. Hopefully this should help the
>>> TripleO team :)
>>>
>>> Reach out if you have any questions at all :)
>>>
>>> Thanks,
>>>
>>> Alex
>>>
>>> On 3/13/17, 10:32 PM, "Emilien Macchi" <emil...@redhat.com> wrote:
>>>
>>> Team,
>>>
>>> [adding Alexandra, OpenStack Docs PTL]
>>>
>>> It seems like there is a common interest in pushing deployment guides
>>> for different OpenStack Deployment projects: OSA, Kolla.
>>> The landing page is here:
>>> https://docs.openstack.org/project-deploy-guide/newton/
>>>
>>> And one example:
>>>
>>> https://docs.openstack.org/project-deploy-guide/openstack-ansible/newton/
>>>
>>> I think this is pretty awesome and it would bring more visibility for
>>> TripleO project, and help our community to find TripleO documentation
>>> from a consistent place.
>>>
>>> The good news, is that openstack-docs team built a pretty solid
>>>     workflow to make that happen:
>>>
>>> https://docs.openstack.org/contributor-guide/project-deploy-guide.html
>>> And we don't need to create new repos or do any crazy changes. It
>>> would probably be some refactoring and sphinx things.
>>>
>>> Alexandra, please add any words if I missed something obvious.
>>>
>>> Feedback from the team would be welcome here before we engage any
>>> work,
>>>
>>> Thanks!
>>> --
>>> Emilien Macchi
>>>
>>>
>>
>>
>>
>> --
>> Emilien Macchi
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> @flaper87
> Flavio Percoco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] container jobs are unstable

2017-03-21 Thread Emilien Macchi
Hey,

I've noticed that container jobs look pretty unstable lately; to me,
it sounds like a timeout:
http://logs.openstack.org/19/447319/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/bca496a/console.html#_2017-03-22_00_08_55_358973

If anyone could file a bug and see how we can bring it back as soon as
possible, I think we want to maintain this job in stable shape.
I remember Container squad wanted it voting because it was supposed to
be stable, but I'm not sure that's the case today.

Also, it would be great to have the container jobs in
http://tripleo.org/cistatus.html - what do you think?

Thanks for your help,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] propose Alex Schultz core on tripleo-heat-templates

2017-03-21 Thread Emilien Macchi
On Mon, Mar 13, 2017 at 2:26 PM, John Trowbridge <tr...@redhat.com> wrote:
>
>
> On 03/13/2017 10:30 AM, Emilien Macchi wrote:
>> Hi,
>>
>> Alex is already core on instack-undercloud and puppet-tripleo.
>
> +1 it is actually a bit odd to be +2 on puppet-tripleo without being +2
> on THT, since so many changes span the two repos.
>
>

Positive votes and no negative feedback, welcome to THT core!

Thanks Alex for your hard work,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


<    1   2   3   4   5   6   7   8   9   10   >