Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-10 Thread Wesley Hayutin
On Wed, Nov 8, 2017 at 5:10 PM, Wesley Hayutin  wrote:

>
>
> On Wed, Nov 8, 2017 at 5:00 PM, Alex Schultz  wrote:
>
>> On Tue, Nov 7, 2017 at 2:59 PM, Emilien Macchi 
>> wrote:
>> > On Wed, Nov 8, 2017 at 3:30 AM, James Slagle 
>> wrote:
>> >> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi 
>> wrote:
>> >>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince 
>> wrote:
>> >>> [...]
>> >>>
>>   -CI resources: better use of CI resources. At the PTG we received
>>  feedback from the OpenStack infrastructure team that our upstream CI
>>  resource usage is quite high at times (even as high as 50% of the
>>  total). Because of the shared framework and single node capabilities
>> we
>>  can re-architecture much of our upstream CI matrix around single
>> node.
>>  We no longer require multinode jobs to be able to test many of the
>>  services in tripleo-heat-templates... we can just use a single cloud
>> VM
>>  instead. We'll still want multinode undercloud -> overcloud jobs for
>>  testing things like HA and baremetal provisioning. But we can cover a
>>  large set of the services (in particular many of the new scenario
>> jobs
>>  we added in Pike) with single node CI test runs in much less time.
>> >>>
>> >>> After the last (terrible) weeks in CI, it's pretty clear we need to
>> >>> find a solution to reduce and optimize our testing.
>> >>> I'm now really convinced by switching our current scenarios jobs to
>> >>> NOT deploy the overcloud, and just an undercloud with composable
>> >>> services & run tempest.
>> >>
>> >> +1 if you mean just the scenarios.
>> >
>> > Yes, just scenarios.
>> >
>> >> I think we need to keep at least 1 multinode job voting that deploys
>> >> the overcloud, probably containers-multinode.
>> >
>> > Yes, exactly, and also work on optimizing OVB jobs (maybe just keep
>> > one or 2 jobs, instead 3).
>> >
>> >>> Benefits:
>> >>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>> >>> - faster (no overcloud)
>> >>> - reduce gate queue time, faster development process, faster CI
>> >>>
>> >>> Challenges:
>> >>> - keep overcloud testing, with OVB
>> >>
>> >> This is why I'm not sure what you're proposing. Do you mean switch all
>> >> multinode jobs to be just an undercloud, or just the scenarios?
>> >
>> > Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be
>> > tested with multinode though but well).
>> >
>> >>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>> >>> containerized services on overcloud.
>> >>>
>> >>> I really want to get consensus on these points, please raise your
>> >>> voice now before we engage some work on that front.
>> >>
>> >> I'm fine to optimize the scenarios to be undercloud driven, but feel
>> >> we still need a multinode job that deploys the overcloud in the gate.
>> >> Otherwise, we'll have nothing that deploys an overcloud in the gate,
>> >> which is a step in the wrong direction imo. Primarily, b/c of the loss
>> >> of coverage around mistral and all of our workflows. Perhaps down the
>> >> road we could find ways to optimize that by using an ephemeral Mistral
>> >> (similar to the ephemeral Heat container), and then use a single node,
>> >> but we're not there yet.
>> >>
>> >> On the other hand, if the goal is just to test less upstream so that
>> >> we can more quickly merge code, then *not* deploying an overcloud in
>> >> the gate at all seems to fit that goal. Is that what you're after?
>> >
>> > Yes. Thanks for reformulate with better words.
>> > Just to be clear, I want to transform the scenarios into single-node
>> > jobs that deploy the SAME services (using composable services) from
>> > the undercloud, using the new ansible installer. I also want to keep
>> > running Tempest.
>> > And of course, like we said, keep one multinode job to test overcloud
>> > workflow, and OVB with some adjustments.
>> >
>>
>> So I'm ok with switching to use the containerized undercloud deploy to
>> smoke test functionality of more complex openstack service
>> deployments. What I would like to see prior to investing in this is
>> that the plain containerized undercloud deploy job reliability is on
>> par with the existing undercloud install.  We had to switch the
>> undercloud-containers back to non-voting due to higher failure rates
>> and it is still not voting.
>
>
> Agree, once we have a little success I'll update featureset027 ( the
> undercloud-containers )
> which is still non-voting to use this updated containerized deployment.
> Then we'll compare
> the undercloud-oooq to undercloud-containers (fs027) After a few weeks of
> running.
>
>
>> With the current state of CI being
>> questionable due to random failures which are not fully have resolved,
>> I would prefer that we ensure existing CI is stable and that what we
>> plan to move is as stable.
>>
>
> 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-08 Thread Wesley Hayutin
On Wed, Nov 8, 2017 at 5:00 PM, Alex Schultz  wrote:

> On Tue, Nov 7, 2017 at 2:59 PM, Emilien Macchi  wrote:
> > On Wed, Nov 8, 2017 at 3:30 AM, James Slagle 
> wrote:
> >> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi 
> wrote:
> >>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
> >>> [...]
> >>>
>   -CI resources: better use of CI resources. At the PTG we received
>  feedback from the OpenStack infrastructure team that our upstream CI
>  resource usage is quite high at times (even as high as 50% of the
>  total). Because of the shared framework and single node capabilities
> we
>  can re-architecture much of our upstream CI matrix around single node.
>  We no longer require multinode jobs to be able to test many of the
>  services in tripleo-heat-templates... we can just use a single cloud
> VM
>  instead. We'll still want multinode undercloud -> overcloud jobs for
>  testing things like HA and baremetal provisioning. But we can cover a
>  large set of the services (in particular many of the new scenario jobs
>  we added in Pike) with single node CI test runs in much less time.
> >>>
> >>> After the last (terrible) weeks in CI, it's pretty clear we need to
> >>> find a solution to reduce and optimize our testing.
> >>> I'm now really convinced by switching our current scenarios jobs to
> >>> NOT deploy the overcloud, and just an undercloud with composable
> >>> services & run tempest.
> >>
> >> +1 if you mean just the scenarios.
> >
> > Yes, just scenarios.
> >
> >> I think we need to keep at least 1 multinode job voting that deploys
> >> the overcloud, probably containers-multinode.
> >
> > Yes, exactly, and also work on optimizing OVB jobs (maybe just keep
> > one or 2 jobs, instead 3).
> >
> >>> Benefits:
> >>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
> >>> - faster (no overcloud)
> >>> - reduce gate queue time, faster development process, faster CI
> >>>
> >>> Challenges:
> >>> - keep overcloud testing, with OVB
> >>
> >> This is why I'm not sure what you're proposing. Do you mean switch all
> >> multinode jobs to be just an undercloud, or just the scenarios?
> >
> > Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be
> > tested with multinode though but well).
> >
> >>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
> >>> containerized services on overcloud.
> >>>
> >>> I really want to get consensus on these points, please raise your
> >>> voice now before we engage some work on that front.
> >>
> >> I'm fine to optimize the scenarios to be undercloud driven, but feel
> >> we still need a multinode job that deploys the overcloud in the gate.
> >> Otherwise, we'll have nothing that deploys an overcloud in the gate,
> >> which is a step in the wrong direction imo. Primarily, b/c of the loss
> >> of coverage around mistral and all of our workflows. Perhaps down the
> >> road we could find ways to optimize that by using an ephemeral Mistral
> >> (similar to the ephemeral Heat container), and then use a single node,
> >> but we're not there yet.
> >>
> >> On the other hand, if the goal is just to test less upstream so that
> >> we can more quickly merge code, then *not* deploying an overcloud in
> >> the gate at all seems to fit that goal. Is that what you're after?
> >
> > Yes. Thanks for reformulate with better words.
> > Just to be clear, I want to transform the scenarios into single-node
> > jobs that deploy the SAME services (using composable services) from
> > the undercloud, using the new ansible installer. I also want to keep
> > running Tempest.
> > And of course, like we said, keep one multinode job to test overcloud
> > workflow, and OVB with some adjustments.
> >
>
> So I'm ok with switching to use the containerized undercloud deploy to
> smoke test functionality of more complex openstack service
> deployments. What I would like to see prior to investing in this is
> that the plain containerized undercloud deploy job reliability is on
> par with the existing undercloud install.  We had to switch the
> undercloud-containers back to non-voting due to higher failure rates
> and it is still not voting.


Agree, once we have a little success I'll update featureset027 ( the
undercloud-containers )
which is still non-voting to use this updated containerized deployment.
Then we'll compare
the undercloud-oooq to undercloud-containers (fs027) After a few weeks of
running.


> With the current state of CI being
> questionable due to random failures which are not fully have resolved,
> I would prefer that we ensure existing CI is stable and that what we
> plan to move is as stable.
>

Agreed,
There are times IMHO when one must strike when the iron is hot on certain
parts of the work here.  I felt compelled to help bootstrap Ian with the
containerized undercloud work or see old habits remain and 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-08 Thread Alex Schultz
On Tue, Nov 7, 2017 at 2:59 PM, Emilien Macchi  wrote:
> On Wed, Nov 8, 2017 at 3:30 AM, James Slagle  wrote:
>> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi  wrote:
>>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
>>> [...]
>>>
  -CI resources: better use of CI resources. At the PTG we received
 feedback from the OpenStack infrastructure team that our upstream CI
 resource usage is quite high at times (even as high as 50% of the
 total). Because of the shared framework and single node capabilities we
 can re-architecture much of our upstream CI matrix around single node.
 We no longer require multinode jobs to be able to test many of the
 services in tripleo-heat-templates... we can just use a single cloud VM
 instead. We'll still want multinode undercloud -> overcloud jobs for
 testing things like HA and baremetal provisioning. But we can cover a
 large set of the services (in particular many of the new scenario jobs
 we added in Pike) with single node CI test runs in much less time.
>>>
>>> After the last (terrible) weeks in CI, it's pretty clear we need to
>>> find a solution to reduce and optimize our testing.
>>> I'm now really convinced by switching our current scenarios jobs to
>>> NOT deploy the overcloud, and just an undercloud with composable
>>> services & run tempest.
>>
>> +1 if you mean just the scenarios.
>
> Yes, just scenarios.
>
>> I think we need to keep at least 1 multinode job voting that deploys
>> the overcloud, probably containers-multinode.
>
> Yes, exactly, and also work on optimizing OVB jobs (maybe just keep
> one or 2 jobs, instead 3).
>
>>> Benefits:
>>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>>> - faster (no overcloud)
>>> - reduce gate queue time, faster development process, faster CI
>>>
>>> Challenges:
>>> - keep overcloud testing, with OVB
>>
>> This is why I'm not sure what you're proposing. Do you mean switch all
>> multinode jobs to be just an undercloud, or just the scenarios?
>
> Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be
> tested with multinode though but well).
>
>>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>>> containerized services on overcloud.
>>>
>>> I really want to get consensus on these points, please raise your
>>> voice now before we engage some work on that front.
>>
>> I'm fine to optimize the scenarios to be undercloud driven, but feel
>> we still need a multinode job that deploys the overcloud in the gate.
>> Otherwise, we'll have nothing that deploys an overcloud in the gate,
>> which is a step in the wrong direction imo. Primarily, b/c of the loss
>> of coverage around mistral and all of our workflows. Perhaps down the
>> road we could find ways to optimize that by using an ephemeral Mistral
>> (similar to the ephemeral Heat container), and then use a single node,
>> but we're not there yet.
>>
>> On the other hand, if the goal is just to test less upstream so that
>> we can more quickly merge code, then *not* deploying an overcloud in
>> the gate at all seems to fit that goal. Is that what you're after?
>
> Yes. Thanks for reformulate with better words.
> Just to be clear, I want to transform the scenarios into single-node
> jobs that deploy the SAME services (using composable services) from
> the undercloud, using the new ansible installer. I also want to keep
> running Tempest.
> And of course, like we said, keep one multinode job to test overcloud
> workflow, and OVB with some adjustments.
>

So I'm ok with switching to use the containerized undercloud deploy to
smoke test functionality of more complex openstack service
deployments. What I would like to see prior to investing in this is
that the plain containerized undercloud deploy job reliability is on
par with the existing undercloud install.  We had to switch the
undercloud-containers back to non-voting due to higher failure rates
and it is still not voting.  With the current state of CI being
questionable due to random failures which are not fully have resolved,
I would prefer that we ensure existing CI is stable and that what we
plan to move is as stable.

Additionally I think we need to ensure that the ovb jobs that still do
full deployment process become voting by switching to 3rd party CI.

Thanks,
-Alex

> Is it good?
>
> Thanks,
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-08 Thread Wesley Hayutin
On Tue, Nov 7, 2017 at 9:07 PM, James Slagle  wrote:

> On Tue, Nov 7, 2017 at 4:59 PM, Emilien Macchi  wrote:
> > Yes. Thanks for reformulate with better words.
> > Just to be clear, I want to transform the scenarios into single-node
> > jobs that deploy the SAME services (using composable services) from
> > the undercloud, using the new ansible installer. I also want to keep
> > running Tempest.
> > And of course, like we said, keep one multinode job to test overcloud
> > workflow, and OVB with some adjustments.
> >
> > Is it good?
>
> +1
>
>
> --
> -- James Slagle
> --
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


FYI,
Slow and I have been pair programming to get the latest containerized
undercloud work into the mainline CI.
The old containerized undercloud work in featureset027 was working and not
to be confused with this attempt.

Here's a link
http://logs.openstack.org/18/518118/2/check/legacy-tripleo-ci-centos-7-undercloud-containers/4eef40f/logs/undercloud/home/zuul/undercloud_install.log.txt.gz
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-07 Thread James Slagle
On Tue, Nov 7, 2017 at 4:59 PM, Emilien Macchi  wrote:
> Yes. Thanks for reformulate with better words.
> Just to be clear, I want to transform the scenarios into single-node
> jobs that deploy the SAME services (using composable services) from
> the undercloud, using the new ansible installer. I also want to keep
> running Tempest.
> And of course, like we said, keep one multinode job to test overcloud
> workflow, and OVB with some adjustments.
>
> Is it good?

+1


-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-07 Thread Emilien Macchi
On Wed, Nov 8, 2017 at 3:30 AM, James Slagle  wrote:
> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi  wrote:
>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
>> [...]
>>
>>>  -CI resources: better use of CI resources. At the PTG we received
>>> feedback from the OpenStack infrastructure team that our upstream CI
>>> resource usage is quite high at times (even as high as 50% of the
>>> total). Because of the shared framework and single node capabilities we
>>> can re-architecture much of our upstream CI matrix around single node.
>>> We no longer require multinode jobs to be able to test many of the
>>> services in tripleo-heat-templates... we can just use a single cloud VM
>>> instead. We'll still want multinode undercloud -> overcloud jobs for
>>> testing things like HA and baremetal provisioning. But we can cover a
>>> large set of the services (in particular many of the new scenario jobs
>>> we added in Pike) with single node CI test runs in much less time.
>>
>> After the last (terrible) weeks in CI, it's pretty clear we need to
>> find a solution to reduce and optimize our testing.
>> I'm now really convinced by switching our current scenarios jobs to
>> NOT deploy the overcloud, and just an undercloud with composable
>> services & run tempest.
>
> +1 if you mean just the scenarios.

Yes, just scenarios.

> I think we need to keep at least 1 multinode job voting that deploys
> the overcloud, probably containers-multinode.

Yes, exactly, and also work on optimizing OVB jobs (maybe just keep
one or 2 jobs, instead 3).

>> Benefits:
>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>> - faster (no overcloud)
>> - reduce gate queue time, faster development process, faster CI
>>
>> Challenges:
>> - keep overcloud testing, with OVB
>
> This is why I'm not sure what you're proposing. Do you mean switch all
> multinode jobs to be just an undercloud, or just the scenarios?

Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be
tested with multinode though but well).

>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>> containerized services on overcloud.
>>
>> I really want to get consensus on these points, please raise your
>> voice now before we engage some work on that front.
>
> I'm fine to optimize the scenarios to be undercloud driven, but feel
> we still need a multinode job that deploys the overcloud in the gate.
> Otherwise, we'll have nothing that deploys an overcloud in the gate,
> which is a step in the wrong direction imo. Primarily, b/c of the loss
> of coverage around mistral and all of our workflows. Perhaps down the
> road we could find ways to optimize that by using an ephemeral Mistral
> (similar to the ephemeral Heat container), and then use a single node,
> but we're not there yet.
>
> On the other hand, if the goal is just to test less upstream so that
> we can more quickly merge code, then *not* deploying an overcloud in
> the gate at all seems to fit that goal. Is that what you're after?

Yes. Thanks for reformulate with better words.
Just to be clear, I want to transform the scenarios into single-node
jobs that deploy the SAME services (using composable services) from
the undercloud, using the new ansible installer. I also want to keep
running Tempest.
And of course, like we said, keep one multinode job to test overcloud
workflow, and OVB with some adjustments.

Is it good?

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-07 Thread James Slagle
On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi  wrote:
> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
> [...]
>
>>  -CI resources: better use of CI resources. At the PTG we received
>> feedback from the OpenStack infrastructure team that our upstream CI
>> resource usage is quite high at times (even as high as 50% of the
>> total). Because of the shared framework and single node capabilities we
>> can re-architecture much of our upstream CI matrix around single node.
>> We no longer require multinode jobs to be able to test many of the
>> services in tripleo-heat-templates... we can just use a single cloud VM
>> instead. We'll still want multinode undercloud -> overcloud jobs for
>> testing things like HA and baremetal provisioning. But we can cover a
>> large set of the services (in particular many of the new scenario jobs
>> we added in Pike) with single node CI test runs in much less time.
>
> After the last (terrible) weeks in CI, it's pretty clear we need to
> find a solution to reduce and optimize our testing.
> I'm now really convinced by switching our current scenarios jobs to
> NOT deploy the overcloud, and just an undercloud with composable
> services & run tempest.

+1 if you mean just the scenarios.

I think we need to keep at least 1 multinode job voting that deploys
the overcloud, probably containers-multinode.

> Benefits:
> - deploy 1 node instead of 2 nodes, so we save nodepool resources
> - faster (no overcloud)
> - reduce gate queue time, faster development process, faster CI
>
> Challenges:
> - keep overcloud testing, with OVB

This is why I'm not sure what you're proposing. Do you mean switch all
multinode jobs to be just an undercloud, or just the scenarios?

> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
> containerized services on overcloud.
>
> I really want to get consensus on these points, please raise your
> voice now before we engage some work on that front.

I'm fine to optimize the scenarios to be undercloud driven, but feel
we still need a multinode job that deploys the overcloud in the gate.
Otherwise, we'll have nothing that deploys an overcloud in the gate,
which is a step in the wrong direction imo. Primarily, b/c of the loss
of coverage around mistral and all of our workflows. Perhaps down the
road we could find ways to optimize that by using an ephemeral Mistral
(similar to the ephemeral Heat container), and then use a single node,
but we're not there yet.

On the other hand, if the goal is just to test less upstream so that
we can more quickly merge code, then *not* deploying an overcloud in
the gate at all seems to fit that goal. Is that what you're after?

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-06 Thread Emilien Macchi
I took an action to remove scenarios/baremetal jobs on pike/master:
https://review.openstack.org/518210

I think it's a good step forward cleaning things up and saving CI resources.
I also think we should keep one multinode/baremetal job on pike (and
probably ovb), and kill all baremetal jobs in master. That would be
the next iteration I guess.

Any feedback is welcome,

On Mon, Nov 6, 2017 at 6:22 PM, Emilien Macchi  wrote:
> On Mon, Nov 6, 2017 at 12:57 PM, Wesley Hayutin  wrote:
>>
>>
>> On Mon, Nov 6, 2017 at 7:35 AM, Bogdan Dobrelya  wrote:
>>>
>>> On 11/6/17 1:01 AM, Emilien Macchi wrote:

 On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
 [...]

>   -CI resources: better use of CI resources. At the PTG we received
> feedback from the OpenStack infrastructure team that our upstream CI
> resource usage is quite high at times (even as high as 50% of the
> total). Because of the shared framework and single node capabilities we
> can re-architecture much of our upstream CI matrix around single node.
> We no longer require multinode jobs to be able to test many of the
> services in tripleo-heat-templates... we can just use a single cloud VM
> instead. We'll still want multinode undercloud -> overcloud jobs for
> testing things like HA and baremetal provisioning. But we can cover a
> large set of the services (in particular many of the new scenario jobs
> we added in Pike) with single node CI test runs in much less time.


 After the last (terrible) weeks in CI, it's pretty clear we need to
 find a solution to reduce and optimize our testing.
 I'm now really convinced by switching our current scenarios jobs to
 NOT deploy the overcloud, and just an undercloud with composable
 services & run tempest.
>>>
>>>
>>> +1
>>> And we should start using the quickstart-extras undercloud-reploy role for
>>> that.
>>>

 Benefits:
 - deploy 1 node instead of 2 nodes, so we save nodepool resources
 - faster (no overcloud)
 - reduce gate queue time, faster development process, faster CI

 Challenges:
 - keep overcloud testing, with OVB
 - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
 containerized services on overcloud.

 I really want to get consensus on these points, please raise your
 voice now before we engage some work on that front.

 [...]

>>>
>>>
>>> --
>>> Best regards,
>>> Bogdan Dobrelya,
>>> Irc #bogdando
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> OK,
>> Just got off the containers call.  We discussed the CI requirements for the
>> containerized undercloud.
>>
>> In the upstream, launched via quickstart not tripleo.sh we want to see
>>
>> 1) undercloud-containers - a containerized install, should be voting by m1
>
> Hum. We're already in m2 now FYI.
>
>> 2) undercloud-containers-update - minor updates run on containerized
>> underclouds, should be voting by m2
>> 3) undercloud-containers-upgrade - major upgrade from
>> non-containerized to containerized undercloud, should be voting by m2.
>>
>> The above three items will enable us to test the quality of just the
>> undercloud install.
>>
>> Ian and I are also working together on testing full deployments with the
>> containerized
>> undercloud to test how stable full runs are generally.  This will
>> help us assess the readiness of switching over in full in queens.
>>
>> This will also then lead into discussions and planning around where we can
>> remove
>> multinode testing in upstream and start to fully utilize the benefits of the
>> containerized undercloud.
>>
>> Please contact myself or Sagi regarding changes in the CI for the
>> undercloud.
>> Thanks
>
> I did this patch:
> https://review.openstack.org/518197
>
> So we can switch our non voting job to use the quickstart featureset.
> Once the switch is made, we need to make sure the job actually works
> fine, otherwise we'll have to make adjustments.
>
> Next iterations in my mind:
> - run undercloud sanity test on undercloud-container job
> - switch existing featureset for scenarios to only deploy a
> containerized undercloud & run Tempest. The only blocker I see for
> that is the fact scenarios are multinode, and we now want one node.
>   2 options:
> - switch scenarios iteratively and when one works, patch infra to
> switch the job to 1node.
> - (expensive) create experimental jobs for each scenarios (and
> featuresets...) and make the switch at some point.
>
> I have a preference for option 1 which sounds easier and faster.
>
> Do we have an etherpad where we can collaborate and list 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-06 Thread Emilien Macchi
On Mon, Nov 6, 2017 at 12:57 PM, Wesley Hayutin  wrote:
>
>
> On Mon, Nov 6, 2017 at 7:35 AM, Bogdan Dobrelya  wrote:
>>
>> On 11/6/17 1:01 AM, Emilien Macchi wrote:
>>>
>>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
>>> [...]
>>>
   -CI resources: better use of CI resources. At the PTG we received
 feedback from the OpenStack infrastructure team that our upstream CI
 resource usage is quite high at times (even as high as 50% of the
 total). Because of the shared framework and single node capabilities we
 can re-architecture much of our upstream CI matrix around single node.
 We no longer require multinode jobs to be able to test many of the
 services in tripleo-heat-templates... we can just use a single cloud VM
 instead. We'll still want multinode undercloud -> overcloud jobs for
 testing things like HA and baremetal provisioning. But we can cover a
 large set of the services (in particular many of the new scenario jobs
 we added in Pike) with single node CI test runs in much less time.
>>>
>>>
>>> After the last (terrible) weeks in CI, it's pretty clear we need to
>>> find a solution to reduce and optimize our testing.
>>> I'm now really convinced by switching our current scenarios jobs to
>>> NOT deploy the overcloud, and just an undercloud with composable
>>> services & run tempest.
>>
>>
>> +1
>> And we should start using the quickstart-extras undercloud-reploy role for
>> that.
>>
>>>
>>> Benefits:
>>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>>> - faster (no overcloud)
>>> - reduce gate queue time, faster development process, faster CI
>>>
>>> Challenges:
>>> - keep overcloud testing, with OVB
>>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>>> containerized services on overcloud.
>>>
>>> I really want to get consensus on these points, please raise your
>>> voice now before we engage some work on that front.
>>>
>>> [...]
>>>
>>
>>
>> --
>> Best regards,
>> Bogdan Dobrelya,
>> Irc #bogdando
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> OK,
> Just got off the containers call.  We discussed the CI requirements for the
> containerized undercloud.
>
> In the upstream, launched via quickstart not tripleo.sh we want to see
>
> 1) undercloud-containers - a containerized install, should be voting by m1

Hum. We're already in m2 now FYI.

> 2) undercloud-containers-update - minor updates run on containerized
> underclouds, should be voting by m2
> 3) undercloud-containers-upgrade - major upgrade from
> non-containerized to containerized undercloud, should be voting by m2.
>
> The above three items will enable us to test the quality of just the
> undercloud install.
>
> Ian and I are also working together on testing full deployments with the
> containerized
> undercloud to test how stable full runs are generally.  This will
> help us assess the readiness of switching over in full in queens.
>
> This will also then lead into discussions and planning around where we can
> remove
> multinode testing in upstream and start to fully utilize the benefits of the
> containerized undercloud.
>
> Please contact myself or Sagi regarding changes in the CI for the
> undercloud.
> Thanks

I did this patch:
https://review.openstack.org/518197

So we can switch our non voting job to use the quickstart featureset.
Once the switch is made, we need to make sure the job actually works
fine, otherwise we'll have to make adjustments.

Next iterations in my mind:
- run undercloud sanity test on undercloud-container job
- switch existing featureset for scenarios to only deploy a
containerized undercloud & run Tempest. The only blocker I see for
that is the fact scenarios are multinode, and we now want one node.
  2 options:
- switch scenarios iteratively and when one works, patch infra to
switch the job to 1node.
- (expensive) create experimental jobs for each scenarios (and
featuresets...) and make the switch at some point.

I have a preference for option 1 which sounds easier and faster.

Do we have an etherpad where we can collaborate and list tasks &
assign folks? I don't want to overlap with any ongoing effort. If not,
let's create one.

Thanks Wes for your help, it's really cool to see we're making progress here.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-06 Thread Wesley Hayutin
On Mon, Nov 6, 2017 at 7:35 AM, Bogdan Dobrelya  wrote:

> On 11/6/17 1:01 AM, Emilien Macchi wrote:
>
>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
>> [...]
>>
>>   -CI resources: better use of CI resources. At the PTG we received
>>> feedback from the OpenStack infrastructure team that our upstream CI
>>> resource usage is quite high at times (even as high as 50% of the
>>> total). Because of the shared framework and single node capabilities we
>>> can re-architecture much of our upstream CI matrix around single node.
>>> We no longer require multinode jobs to be able to test many of the
>>> services in tripleo-heat-templates... we can just use a single cloud VM
>>> instead. We'll still want multinode undercloud -> overcloud jobs for
>>> testing things like HA and baremetal provisioning. But we can cover a
>>> large set of the services (in particular many of the new scenario jobs
>>> we added in Pike) with single node CI test runs in much less time.
>>>
>>
>> After the last (terrible) weeks in CI, it's pretty clear we need to
>> find a solution to reduce and optimize our testing.
>> I'm now really convinced by switching our current scenarios jobs to
>> NOT deploy the overcloud, and just an undercloud with composable
>> services & run tempest.
>>
>
> +1
> And we should start using the quickstart-extras undercloud-reploy role for
> that.
>
>
>> Benefits:
>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>> - faster (no overcloud)
>> - reduce gate queue time, faster development process, faster CI
>>
>> Challenges:
>> - keep overcloud testing, with OVB
>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>> containerized services on overcloud.
>>
>> I really want to get consensus on these points, please raise your
>> voice now before we engage some work on that front.
>>
>> [...]
>>
>>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

OK,
Just got off the containers call.  We discussed the CI requirements for the
containerized undercloud.

In the upstream, launched via quickstart not tripleo.sh we want to see

1) undercloud-containers - a containerized install, should be voting by m1
2) undercloud-containers-update - minor updates run on containerized
underclouds, should be voting by m2
3) undercloud-containers-upgrade - major upgrade from
non-containerized to containerized undercloud, should be voting by m2.

The above three items will enable us to test the quality of just the
undercloud install.

Ian and I are also working together on testing full deployments with the
containerized
undercloud to test how stable full runs are generally.  This will
help us assess the readiness of switching over in full in queens.

This will also then lead into discussions and planning around where we can
remove
multinode testing in upstream and start to fully utilize the benefits of
the containerized undercloud.

Please contact myself or Sagi regarding changes in the CI for the
undercloud.
Thanks
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-06 Thread Emilien Macchi
On Mon, Nov 6, 2017 at 4:35 AM, Bogdan Dobrelya  wrote:
> On 11/6/17 1:01 AM, Emilien Macchi wrote:
>>
>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
>> [...]
>>
>>>   -CI resources: better use of CI resources. At the PTG we received
>>> feedback from the OpenStack infrastructure team that our upstream CI
>>> resource usage is quite high at times (even as high as 50% of the
>>> total). Because of the shared framework and single node capabilities we
>>> can re-architecture much of our upstream CI matrix around single node.
>>> We no longer require multinode jobs to be able to test many of the
>>> services in tripleo-heat-templates... we can just use a single cloud VM
>>> instead. We'll still want multinode undercloud -> overcloud jobs for
>>> testing things like HA and baremetal provisioning. But we can cover a
>>> large set of the services (in particular many of the new scenario jobs
>>> we added in Pike) with single node CI test runs in much less time.
>>
>>
>> After the last (terrible) weeks in CI, it's pretty clear we need to
>> find a solution to reduce and optimize our testing.
>> I'm now really convinced by switching our current scenarios jobs to
>> NOT deploy the overcloud, and just an undercloud with composable
>> services & run tempest.
>
>
> +1
> And we should start using the quickstart-extras undercloud-reploy role for
> that.

YES! and reflect what our users would do in real life. No workaround.

>>
>> Benefits:
>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>> - faster (no overcloud)
>> - reduce gate queue time, faster development process, faster CI
>>
>> Challenges:
>> - keep overcloud testing, with OVB
>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>> containerized services on overcloud.
>>
>> I really want to get consensus on these points, please raise your
>> voice now before we engage some work on that front.
>>
>> [...]
>>
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-06 Thread Bogdan Dobrelya

On 11/6/17 1:01 AM, Emilien Macchi wrote:

On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
[...]


  -CI resources: better use of CI resources. At the PTG we received
feedback from the OpenStack infrastructure team that our upstream CI
resource usage is quite high at times (even as high as 50% of the
total). Because of the shared framework and single node capabilities we
can re-architecture much of our upstream CI matrix around single node.
We no longer require multinode jobs to be able to test many of the
services in tripleo-heat-templates... we can just use a single cloud VM
instead. We'll still want multinode undercloud -> overcloud jobs for
testing things like HA and baremetal provisioning. But we can cover a
large set of the services (in particular many of the new scenario jobs
we added in Pike) with single node CI test runs in much less time.


After the last (terrible) weeks in CI, it's pretty clear we need to
find a solution to reduce and optimize our testing.
I'm now really convinced by switching our current scenarios jobs to
NOT deploy the overcloud, and just an undercloud with composable
services & run tempest.


+1
And we should start using the quickstart-extras undercloud-reploy role 
for that.




Benefits:
- deploy 1 node instead of 2 nodes, so we save nodepool resources
- faster (no overcloud)
- reduce gate queue time, faster development process, faster CI

Challenges:
- keep overcloud testing, with OVB
- reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
containerized services on overcloud.

I really want to get consensus on these points, please raise your
voice now before we engage some work on that front.

[...]




--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-05 Thread Wesley Hayutin
On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi  wrote:

> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
> [...]
>
> >  -CI resources: better use of CI resources. At the PTG we received
> > feedback from the OpenStack infrastructure team that our upstream CI
> > resource usage is quite high at times (even as high as 50% of the
> > total). Because of the shared framework and single node capabilities we
> > can re-architecture much of our upstream CI matrix around single node.
> > We no longer require multinode jobs to be able to test many of the
> > services in tripleo-heat-templates... we can just use a single cloud VM
> > instead. We'll still want multinode undercloud -> overcloud jobs for
> > testing things like HA and baremetal provisioning. But we can cover a
> > large set of the services (in particular many of the new scenario jobs
> > we added in Pike) with single node CI test runs in much less time.
>
> After the last (terrible) weeks in CI, it's pretty clear we need to
> find a solution to reduce and optimize our testing.
> I'm now really convinced by switching our current scenarios jobs to
> NOT deploy the overcloud, and just an undercloud with composable
> services & run tempest.
>

First off, I'm really pleased that the containerized undercloud effort has
been reinvigerated for queens.  The containerized undercloud work has been
awesome, really nice work to everyone involved!!

I totally agree we should be shifting to using the undercloud only and
deploy the services we want need for the scenarios on the single node.

I think we should start putting plans in place to start shifting work to
the undercloud only approach, however I think it is way too early to talk
about not deploying the overcloud in CI.  I'd prefer to take a step by step
phased approach to such a large change.

Really good point Emilien and thanks for raising it!!




>
> Benefits:
> - deploy 1 node instead of 2 nodes, so we save nodepool resources
> - faster (no overcloud)
> - reduce gate queue time, faster development process, faster CI
>
> Challenges:
> - keep overcloud testing, with OVB
> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
> containerized services on overcloud.
>
> I really want to get consensus on these points, please raise your
> voice now before we engage some work on that front.
>
> [...]
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-11-05 Thread Emilien Macchi
On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince  wrote:
[...]

>  -CI resources: better use of CI resources. At the PTG we received
> feedback from the OpenStack infrastructure team that our upstream CI
> resource usage is quite high at times (even as high as 50% of the
> total). Because of the shared framework and single node capabilities we
> can re-architecture much of our upstream CI matrix around single node.
> We no longer require multinode jobs to be able to test many of the
> services in tripleo-heat-templates... we can just use a single cloud VM
> instead. We'll still want multinode undercloud -> overcloud jobs for
> testing things like HA and baremetal provisioning. But we can cover a
> large set of the services (in particular many of the new scenario jobs
> we added in Pike) with single node CI test runs in much less time.

After the last (terrible) weeks in CI, it's pretty clear we need to
find a solution to reduce and optimize our testing.
I'm now really convinced by switching our current scenarios jobs to
NOT deploy the overcloud, and just an undercloud with composable
services & run tempest.

Benefits:
- deploy 1 node instead of 2 nodes, so we save nodepool resources
- faster (no overcloud)
- reduce gate queue time, faster development process, faster CI

Challenges:
- keep overcloud testing, with OVB
- reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
containerized services on overcloud.

I really want to get consensus on these points, please raise your
voice now before we engage some work on that front.

[...]
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-18 Thread milanisko k
út 17. 10. 2017 v 17:14 odesílatel Dan Prince  napsal:

> On Tue, 2017-10-17 at 11:46 +, milanisko k wrote:
> >
> > How about the shared container? Wouldn't it be better not have to
> > rely on t-h-t especially if we're "scheduling" (and probably
> > configuring) the services as a single logical entity?
>
> The containers architecture for Pike and Queens is very much oriented
> around preserving the way we deployed the services already on
> baremetal... but moving them into containers. So for Ironic inspector

we had 2 services (2 systemd scripts) both living in separate
> containers. Do the the shared nature of this architecture with regards
> to network and host access this works fine.
>

Unless new features, such as  the inspector support for routed/relayed
DHCP/PXE traffic (or spine network topology), come  into the question.
For this case, as well as for the HA case (with non-overlapping dnsmasq
DHCP pools), the trick with host access won't work alone anymore as the
dnsmasq and inspector need to change each other's (configuration) state. I
guess that old patch needs to address this somehow.


> In the future as we move towards Kubernetes rearchitecting the services
> so they work better in containers is totally fine. If the services are
> that tightly coupled then why not just have one launch the other?


That's my point of view as well.

Then they could live in the single container and have a common launch point.
>

What I'd like to achieve with the supervisord inside of the shared
container as, besides other things, inspector and dnsmasq have to
start/fail together in the HA and spine-leaf-support case.


> Seems fine to me, but certainly isn't a requirement to get these up and
> running in the current architecture.
>

But if addressed right now would save some effort in the future while, as a
bonus, getting us the cool features sooner.

Would you mind testing the containerised undercloud with the inspector
dnsmasq PXE filter patch chain[1] applied?


Thanks,
milan

[1]
https://review.openstack.org/#/q/topic:rebasing_filter+(status:open+OR+status:merged)



>
>
> > Also would allow us to get rid of iptables and better encapsulate the
> > inspector services.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-17 Thread Dan Prince
On Tue, 2017-10-17 at 11:46 +, milanisko k wrote:
> 
> How about the shared container? Wouldn't it be better not have to
> rely on t-h-t especially if we're "scheduling" (and probably
> configuring) the services as a single logical entity? 

The containers architecture for Pike and Queens is very much oriented
around preserving the way we deployed the services already on
baremetal... but moving them into containers. So for Ironic inspector
we had 2 services (2 systemd scripts) both living in separate
containers. Do the the shared nature of this architecture with regards
to network and host access this works fine.

In the future as we move towards Kubernetes rearchitecting the services
so they work better in containers is totally fine. If the services are
that tightly coupled then why not just have one launch the other? Then
they could live in the single container and have a common launch point.
Seems fine to me, but certainly isn't a requirement to get these up and
running in the current architecture.


> Also would allow us to get rid of iptables and better encapsulate the
> inspector services.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-17 Thread milanisko k
út 17. 10. 2017 v 13:06 odesílatel Dan Prince  napsal:

> On Tue, 2017-10-17 at 10:06 +, milanisko k wrote:
> >
> > Does it mean dnsmasq was run from a stand-alone container?
>
> Yes. There are separate containers for the ironic-inspector and
> dnsmasq.
>
> >
> > Could you please point me (in the patch probably) to the spot where
> > we configure inspector container to be able to talk to the iptables
> > to filter the DHCP traffic for dnsmasq?
>
> Both services (ironic-inspector and dnsmasq) are using --net=host and
> --privileged. This essentially has them on the same shared host network
> thus the services can interact with the same iptables rules.
>
> >
> > I guess this configuration binds the dnsmasq container to be
> > "scheduled" together with inspector container on the same node
> > (because of the iptables).
>
> Both services are controlled via the same Heat template and as such
> even though they are in separate containers we can guarantee they
> should always get launched on the same machine.
>

How about the shared container? Wouldn't it be better not have to rely on
t-h-t especially if we're "scheduling" (and probably configuring) the
services as a single logical entity? Also would allow us to get rid of
iptables and better encapsulate the inspector services.

--
milan


> Dan
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-17 Thread Dan Prince
On Tue, 2017-10-17 at 10:06 +, milanisko k wrote:
> 
> Does it mean dnsmasq was run from a stand-alone container?

Yes. There are separate containers for the ironic-inspector and
dnsmasq.

> 
> Could you please point me (in the patch probably) to the spot where
> we configure inspector container to be able to talk to the iptables
> to filter the DHCP traffic for dnsmasq?

Both services (ironic-inspector and dnsmasq) are using --net=host and
--privileged. This essentially has them on the same shared host network
thus the services can interact with the same iptables rules.

> 
> I guess this configuration binds the dnsmasq container to be
> "scheduled" together with inspector container on the same node
> (because of the iptables).

Both services are controlled via the same Heat template and as such
even though they are in separate containers we can guarantee they
should always get launched on the same machine.

Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-17 Thread milanisko k
Hi Dan!

thanks for the testing! I've got couple of questions...


po 16. 10. 2017 v 20:04 odesílatel Dan Prince  napsal:

> On Wed, 2017-10-04 at 15:10 +0200, Dmitry Tantsur wrote:
> > (top-posting, as it is not a direct response to a specific line)
> >
> > This is your friendly reminder that we're not quite near
> > containerized
> > ironic-inspector. The THT for it has probably never been tested at
> > all, and the
> > iptables magic we do may simply not be containers-compatible. Milan
> > would
> > appreciate any help with his ironic-inspector rework.
>
>
> I spent the time today to test our (very old) ironic-inspector patch
> from Pike.
>
> https://review.openstack.org/#/c/457822/
>
> Aside from one tweak I made to run dnsmasq as root (this is how systemd
> runs this process as well) the service seems to be working perfectly.
> Demo recording of what I did below:
>
> https://asciinema.org/a/wGtvZwE65yoasKrRS8NeGMsrH
>
>
Does it mean dnsmasq was run from a stand-alone container?

Could you please point me (in the patch probably) to the spot where we
configure inspector container to be able to talk to the iptables to filter
the DHCP traffic for dnsmasq?

I guess this configuration binds the dnsmasq container to be "scheduled"
together with inspector container on the same node (because of the
iptables).



> Also, just want to re-interate that the t-h-t architecture is also
> capable as a baremetal installation tool. While I would like to see
> inspector containerized if we really need to run it on baremetal this
> architecture would support that fine. It is the same architecture we
> use for the overcloud and as such it supports mixing and matching
> containers alongside baremetal services.


Nice!


>
> If that doesn't make sense let me just say that whatever you plan on
> doing in Queens to Ironic if you plan on supporting that w/ Puppet on
> instack-undercloud I have no doubts about being able to support it in
> the architecture I'm proposing we adopt here... whether we run the
> service on baremetal or in a container.
>

What I'd vote for would be to pack both inspector and its dnsmasq sidekick
into a single container instead, and adopt the dnsmasq filter[1] instead of
the iptables filter. This will make the inspector a self-contained
service/container one could "schedule" where ever they needed.

Cheers,
milan

[1]
https://review.openstack.org/#/q/topic:rebasing_filter+(status:open+OR+status:merged)



>
> Dan
>
> >
> > Dmitry
> >
> > On 10/04/2017 03:00 PM, Dan Prince wrote:
> > > On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
> > > > On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
> > > > wrote:
> > > > >
> > > > >
> > > > > On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz  > > > > om>
> > > > > wrote:
> > > > > >
> > > > > > On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  > > > > > om>
> > > > > > wrote:
> > > > > > > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
> > > > > > > > Hey Dan,
> > > > > > > >
> > > > > > > > Thanks for sending out a note about this. I have a few
> > > > > > > > questions
> > > > > > > > inline.
> > > > > > > >
> > > > > > > > On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince  > > > > > > > t.co
> > > > > > > > m>
> > > > > > > > wrote:
> > > > > > > > > One of the things the TripleO containers team is
> > > > > > > > > planning
> > > > > > > > > on
> > > > > > > > > tackling
> > > > > > > > > in Queens is fully containerizing the undercloud. At
> > > > > > > > > the
> > > > > > > > > PTG we
> > > > > > > > > created
> > > > > > > > > an etherpad [1] that contains a list of features that
> > > > > > > > > need
> > > > > > > > > to be
> > > > > > > > > implemented to fully replace instack-undercloud.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I know we talked about this at the PTG and I was
> > > > > > > > skeptical
> > > > > > > > that this
> > > > > > > > will land in Queens. With the exception of the
> > > > > > > > Container's
> > > > > > > > team
> > > > > > > > wanting this, I'm not sure there is an actual end user
> > > > > > > > who is
> > > > > > > > looking
> > > > > > > > for the feature so I want to make sure we're not just
> > > > > > > > doing
> > > > > > > > more work
> > > > > > > > because we as developers think it's a good idea.
> > > > > > >
> > > > > > > I've heard from several operators that they were actually
> > > > > > > surprised we
> > > > > > > implemented containers in the Overcloud first. Validating a
> > > > > > > new
> > > > > > > deployment framework on a single node Undercloud (for
> > > > > > > operators) before
> > > > > > > overtaking their entire cloud deployment has a lot of merit
> > > > > > > to
> > > > > > > it IMO.
> > > > > > > When you share the same deployment architecture across the
> > > > > > > overcloud/undercloud it puts us in a better position to
> > > > > > > decide
> > > > > > > where to
> > > > > > > expose new 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-16 Thread Dan Prince
On Wed, 2017-10-04 at 15:10 +0200, Dmitry Tantsur wrote:
> (top-posting, as it is not a direct response to a specific line)
> 
> This is your friendly reminder that we're not quite near
> containerized 
> ironic-inspector. The THT for it has probably never been tested at
> all, and the 
> iptables magic we do may simply not be containers-compatible. Milan
> would 
> appreciate any help with his ironic-inspector rework.


I spent the time today to test our (very old) ironic-inspector patch
from Pike.

https://review.openstack.org/#/c/457822/

Aside from one tweak I made to run dnsmasq as root (this is how systemd
runs this process as well) the service seems to be working perfectly.
Demo recording of what I did below:

https://asciinema.org/a/wGtvZwE65yoasKrRS8NeGMsrH

Also, just want to re-interate that the t-h-t architecture is also
capable as a baremetal installation tool. While I would like to see
inspector containerized if we really need to run it on baremetal this
architecture would support that fine. It is the same architecture we
use for the overcloud and as such it supports mixing and matching
containers alongside baremetal services.

If that doesn't make sense let me just say that whatever you plan on
doing in Queens to Ironic if you plan on supporting that w/ Puppet on
instack-undercloud I have no doubts about being able to support it in
the architecture I'm proposing we adopt here... whether we run the
service on baremetal or in a container.

Dan

> 
> Dmitry
> 
> On 10/04/2017 03:00 PM, Dan Prince wrote:
> > On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
> > > On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
> > > wrote:
> > > > 
> > > > 
> > > > On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz  > > > om>
> > > > wrote:
> > > > > 
> > > > > On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  > > > > om>
> > > > > wrote:
> > > > > > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
> > > > > > > Hey Dan,
> > > > > > > 
> > > > > > > Thanks for sending out a note about this. I have a few
> > > > > > > questions
> > > > > > > inline.
> > > > > > > 
> > > > > > > On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince  > > > > > > t.co
> > > > > > > m>
> > > > > > > wrote:
> > > > > > > > One of the things the TripleO containers team is
> > > > > > > > planning
> > > > > > > > on
> > > > > > > > tackling
> > > > > > > > in Queens is fully containerizing the undercloud. At
> > > > > > > > the
> > > > > > > > PTG we
> > > > > > > > created
> > > > > > > > an etherpad [1] that contains a list of features that
> > > > > > > > need
> > > > > > > > to be
> > > > > > > > implemented to fully replace instack-undercloud.
> > > > > > > > 
> > > > > > > 
> > > > > > > I know we talked about this at the PTG and I was
> > > > > > > skeptical
> > > > > > > that this
> > > > > > > will land in Queens. With the exception of the
> > > > > > > Container's
> > > > > > > team
> > > > > > > wanting this, I'm not sure there is an actual end user
> > > > > > > who is
> > > > > > > looking
> > > > > > > for the feature so I want to make sure we're not just
> > > > > > > doing
> > > > > > > more work
> > > > > > > because we as developers think it's a good idea.
> > > > > > 
> > > > > > I've heard from several operators that they were actually
> > > > > > surprised we
> > > > > > implemented containers in the Overcloud first. Validating a
> > > > > > new
> > > > > > deployment framework on a single node Undercloud (for
> > > > > > operators) before
> > > > > > overtaking their entire cloud deployment has a lot of merit
> > > > > > to
> > > > > > it IMO.
> > > > > > When you share the same deployment architecture across the
> > > > > > overcloud/undercloud it puts us in a better position to
> > > > > > decide
> > > > > > where to
> > > > > > expose new features to operators first (when creating the
> > > > > > undercloud or
> > > > > > overcloud for example).
> > > > > > 
> > > > > > Also, if you read my email again I've explicitly listed the
> > > > > > "Containers" benefit last. While I think moving the
> > > > > > undercloud
> > > > > > to
> > > > > > containers is a great benefit all by itself this is more of
> > > > > > a
> > > > > > "framework alignment" in TripleO and gets us out of
> > > > > > maintaining
> > > > > > huge
> > > > > > amounts of technical debt. Re-using the same framework for
> > > > > > the
> > > > > > undercloud and overcloud has a lot of merit. It effectively
> > > > > > streamlines
> > > > > > the development process for service developers, and 3rd
> > > > > > parties
> > > > > > wishing
> > > > > > to integrate some of their components on a single node. Why
> > > > > > be
> > > > > > forced
> > > > > > to create a multi-node dev environment if you don't have to
> > > > > > (aren't
> > > > > > using HA for example).
> > > > > > 
> > > > > > Lets be honest. While instack-undercloud helped solve the
> > > > > > old
> > > > > > "seed" VM
> > 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-16 Thread Bogdan Dobrelya
On 10/3/17 10:46 PM, Dan Prince wrote:
> 
> 
> This reduces our complexity greatly I think in that once it is completed
> will allow us to eliminate two project (instack and instack-undercloud)
> and the maintenance thereof. Furthermore, as this dovetails nice with
> the Ansible
>  
> 
>  IMHO doit.sh is not acceptable as an undercloud installer and
> this is what I've been trying to point out as the actual impact to the
> end user who has to use this thing.
> 
> 
> doit.sh is an example of where the effort is today. It is essentially
> the same stuff we document online
> here: http://tripleo.org/install/containers_deployment/undercloud.html.
> 
> Similar to quickstart it is just something meant to help you setup a dev
> environment.

Please note that quickstart can "doit.sh" [0] as well and even more :)
Although the undercloud_deploy role needs to be supported in the
quickstart.sh maybe, and its documentation [1] should be explaining the
case better.

Undercloud_install_script and the script template itself fully addresses
the needed flexibility for developers and operators. You can git clone /
pip install things, or do not.

It follows the standard quickstart way, which is jinja-templating bash
scripts in order to provide an operator-ish friendly, and independent
from Ansible et al, way to reproduce the scripted steps. And helps to
auto-generate documentation as well.

[0]
https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/undercloud-deploy/README.md
[1] https://docs.openstack.org/tripleo-quickstart/latest/


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-04 Thread Fox, Kevin M
FYI, a container with net=host runs exactly like it was running outside of a 
container with respect to iptables/networking. So that should not be an issue. 
If it can be done on the host, it should be able to happen in a container.

Thanks,
Kevin

From: Dan Prince [dpri...@redhat.com]
Sent: Wednesday, October 04, 2017 9:50 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [TripleO] containerized undercloud in Queens



On Wed, Oct 4, 2017 at 9:10 AM, Dmitry Tantsur 
<dtant...@redhat.com<mailto:dtant...@redhat.com>> wrote:
(top-posting, as it is not a direct response to a specific line)

This is your friendly reminder that we're not quite near containerized 
ironic-inspector. The THT for it has probably never been tested at all, and the 
iptables magic we do may simply not be containers-compatible. Milan would 
appreciate any help with his ironic-inspector rework.


Thanks Dmitry. Exactly the update I was looking for. Look forward to syncing w/ 
Milan on this.

Dan

Dmitry


On 10/04/2017 03:00 PM, Dan Prince wrote:
On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
<dpri...@redhat.com<mailto:dpri...@redhat.com>>
wrote:


On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz 
<aschu...@redhat.com<mailto:aschu...@redhat.com>>
wrote:

On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince 
<dpri...@redhat.com<mailto:dpri...@redhat.com>>
wrote:
On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
Hey Dan,

Thanks for sending out a note about this. I have a few
questions
inline.

On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
<dpri...@redhat.co<mailto:dpri...@redhat.co>
m>
wrote:
One of the things the TripleO containers team is planning
on
tackling
in Queens is fully containerizing the undercloud. At the
PTG we
created
an etherpad [1] that contains a list of features that need
to be
implemented to fully replace instack-undercloud.


I know we talked about this at the PTG and I was skeptical
that this
will land in Queens. With the exception of the Container's
team
wanting this, I'm not sure there is an actual end user who is
looking
for the feature so I want to make sure we're not just doing
more work
because we as developers think it's a good idea.

I've heard from several operators that they were actually
surprised we
implemented containers in the Overcloud first. Validating a new
deployment framework on a single node Undercloud (for
operators) before
overtaking their entire cloud deployment has a lot of merit to
it IMO.
When you share the same deployment architecture across the
overcloud/undercloud it puts us in a better position to decide
where to
expose new features to operators first (when creating the
undercloud or
overcloud for example).

Also, if you read my email again I've explicitly listed the
"Containers" benefit last. While I think moving the undercloud
to
containers is a great benefit all by itself this is more of a
"framework alignment" in TripleO and gets us out of maintaining
huge
amounts of technical debt. Re-using the same framework for the
undercloud and overcloud has a lot of merit. It effectively
streamlines
the development process for service developers, and 3rd parties
wishing
to integrate some of their components on a single node. Why be
forced
to create a multi-node dev environment if you don't have to
(aren't
using HA for example).

Lets be honest. While instack-undercloud helped solve the old
"seed" VM
issue it was outdated the day it landed upstream. The entire
premise of
the tool is that it uses old style "elements" to create the
undercloud
and we moved away from those as the primary means driving the
creation
of the Overcloud years ago at this point. The new
'undercloud_deploy'
installer gets us back to our roots by once again sharing the
same
architecture to create the over and underclouds. A demo from
long ago
expands on this idea a bit:  https://www.youtube.com/watch?v=y1
qMDLAf26
Q=5s

In short, we aren't just doing more work because developers
think it is
a good idea. This has potential to be one of the most useful
architectural changes in TripleO that we've made in years.
Could
significantly decrease our CI reasources if we use it to
replace the
existing scenarios jobs which take multiple VMs per job. Is a
building
block we could use for other features like and HA undercloud.
And yes,
it does also have a huge impact on developer velocity in that
many of
us already prefer to use the tool as a means of streamlining
our
dev/test cycles to minutes instead of hours. Why spend hours
running
quickstart Ansible scripts when in many cases you can just
doit.sh. htt
ps://github.com/dprince/undercloud_containers/blob/master/doit<http://github.com/dprince/undercloud_containers/blob/master/doit>.
sh


So like I've repeatedly said, I'm not completely against it as I
agree

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-04 Thread Dan Prince
On Wed, Oct 4, 2017 at 9:10 AM, Dmitry Tantsur  wrote:

> (top-posting, as it is not a direct response to a specific line)
>
> This is your friendly reminder that we're not quite near containerized
> ironic-inspector. The THT for it has probably never been tested at all, and
> the iptables magic we do may simply not be containers-compatible. Milan
> would appreciate any help with his ironic-inspector rework.
>
>
Thanks Dmitry. Exactly the update I was looking for. Look forward to
syncing w/ Milan on this.

Dan


> Dmitry
>
>
> On 10/04/2017 03:00 PM, Dan Prince wrote:
>
>> On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
>>
>>> On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
>>> wrote:
>>>


 On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz 
 wrote:

>
> On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince 
> wrote:
>
>> On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
>>
>>> Hey Dan,
>>>
>>> Thanks for sending out a note about this. I have a few
>>> questions
>>> inline.
>>>
>>> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince >> m>
>>> wrote:
>>>
 One of the things the TripleO containers team is planning
 on
 tackling
 in Queens is fully containerizing the undercloud. At the
 PTG we
 created
 an etherpad [1] that contains a list of features that need
 to be
 implemented to fully replace instack-undercloud.


>>> I know we talked about this at the PTG and I was skeptical
>>> that this
>>> will land in Queens. With the exception of the Container's
>>> team
>>> wanting this, I'm not sure there is an actual end user who is
>>> looking
>>> for the feature so I want to make sure we're not just doing
>>> more work
>>> because we as developers think it's a good idea.
>>>
>>
>> I've heard from several operators that they were actually
>> surprised we
>> implemented containers in the Overcloud first. Validating a new
>> deployment framework on a single node Undercloud (for
>> operators) before
>> overtaking their entire cloud deployment has a lot of merit to
>> it IMO.
>> When you share the same deployment architecture across the
>> overcloud/undercloud it puts us in a better position to decide
>> where to
>> expose new features to operators first (when creating the
>> undercloud or
>> overcloud for example).
>>
>> Also, if you read my email again I've explicitly listed the
>> "Containers" benefit last. While I think moving the undercloud
>> to
>> containers is a great benefit all by itself this is more of a
>> "framework alignment" in TripleO and gets us out of maintaining
>> huge
>> amounts of technical debt. Re-using the same framework for the
>> undercloud and overcloud has a lot of merit. It effectively
>> streamlines
>> the development process for service developers, and 3rd parties
>> wishing
>> to integrate some of their components on a single node. Why be
>> forced
>> to create a multi-node dev environment if you don't have to
>> (aren't
>> using HA for example).
>>
>> Lets be honest. While instack-undercloud helped solve the old
>> "seed" VM
>> issue it was outdated the day it landed upstream. The entire
>> premise of
>> the tool is that it uses old style "elements" to create the
>> undercloud
>> and we moved away from those as the primary means driving the
>> creation
>> of the Overcloud years ago at this point. The new
>> 'undercloud_deploy'
>> installer gets us back to our roots by once again sharing the
>> same
>> architecture to create the over and underclouds. A demo from
>> long ago
>> expands on this idea a bit:  https://www.youtube.com/watch?v=y1
>> qMDLAf26
>> Q=5s
>>
>> In short, we aren't just doing more work because developers
>> think it is
>> a good idea. This has potential to be one of the most useful
>> architectural changes in TripleO that we've made in years.
>> Could
>> significantly decrease our CI reasources if we use it to
>> replace the
>> existing scenarios jobs which take multiple VMs per job. Is a
>> building
>> block we could use for other features like and HA undercloud.
>> And yes,
>> it does also have a huge impact on developer velocity in that
>> many of
>> us already prefer to use the tool as a means of streamlining
>> our
>> dev/test cycles to minutes instead of hours. Why spend hours
>> running
>> quickstart Ansible scripts when in many cases you can just
>> doit.sh. htt
>> ps://github.com/dprince/undercloud_containers/blob/master/doit.
>> sh
>>
>>
> So like I've repeatedly said, 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-04 Thread Alex Schultz
On Wed, Oct 4, 2017 at 7:00 AM, Dan Prince  wrote:
> On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
>> On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
>> wrote:
>> >
>> >
>> > On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz 
>> > wrote:
>> > >
>> > > On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince 
>> > > wrote:
>> > > > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
>> > > > > Hey Dan,
>> > > > >
>> > > > > Thanks for sending out a note about this. I have a few
>> > > > > questions
>> > > > > inline.
>> > > > >
>> > > > > On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince > > > > > m>
>> > > > > wrote:
>> > > > > > One of the things the TripleO containers team is planning
>> > > > > > on
>> > > > > > tackling
>> > > > > > in Queens is fully containerizing the undercloud. At the
>> > > > > > PTG we
>> > > > > > created
>> > > > > > an etherpad [1] that contains a list of features that need
>> > > > > > to be
>> > > > > > implemented to fully replace instack-undercloud.
>> > > > > >
>> > > > >
>> > > > > I know we talked about this at the PTG and I was skeptical
>> > > > > that this
>> > > > > will land in Queens. With the exception of the Container's
>> > > > > team
>> > > > > wanting this, I'm not sure there is an actual end user who is
>> > > > > looking
>> > > > > for the feature so I want to make sure we're not just doing
>> > > > > more work
>> > > > > because we as developers think it's a good idea.
>> > > >
>> > > > I've heard from several operators that they were actually
>> > > > surprised we
>> > > > implemented containers in the Overcloud first. Validating a new
>> > > > deployment framework on a single node Undercloud (for
>> > > > operators) before
>> > > > overtaking their entire cloud deployment has a lot of merit to
>> > > > it IMO.
>> > > > When you share the same deployment architecture across the
>> > > > overcloud/undercloud it puts us in a better position to decide
>> > > > where to
>> > > > expose new features to operators first (when creating the
>> > > > undercloud or
>> > > > overcloud for example).
>> > > >
>> > > > Also, if you read my email again I've explicitly listed the
>> > > > "Containers" benefit last. While I think moving the undercloud
>> > > > to
>> > > > containers is a great benefit all by itself this is more of a
>> > > > "framework alignment" in TripleO and gets us out of maintaining
>> > > > huge
>> > > > amounts of technical debt. Re-using the same framework for the
>> > > > undercloud and overcloud has a lot of merit. It effectively
>> > > > streamlines
>> > > > the development process for service developers, and 3rd parties
>> > > > wishing
>> > > > to integrate some of their components on a single node. Why be
>> > > > forced
>> > > > to create a multi-node dev environment if you don't have to
>> > > > (aren't
>> > > > using HA for example).
>> > > >
>> > > > Lets be honest. While instack-undercloud helped solve the old
>> > > > "seed" VM
>> > > > issue it was outdated the day it landed upstream. The entire
>> > > > premise of
>> > > > the tool is that it uses old style "elements" to create the
>> > > > undercloud
>> > > > and we moved away from those as the primary means driving the
>> > > > creation
>> > > > of the Overcloud years ago at this point. The new
>> > > > 'undercloud_deploy'
>> > > > installer gets us back to our roots by once again sharing the
>> > > > same
>> > > > architecture to create the over and underclouds. A demo from
>> > > > long ago
>> > > > expands on this idea a bit:  https://www.youtube.com/watch?v=y1
>> > > > qMDLAf26
>> > > > Q=5s
>> > > >
>> > > > In short, we aren't just doing more work because developers
>> > > > think it is
>> > > > a good idea. This has potential to be one of the most useful
>> > > > architectural changes in TripleO that we've made in years.
>> > > > Could
>> > > > significantly decrease our CI reasources if we use it to
>> > > > replace the
>> > > > existing scenarios jobs which take multiple VMs per job. Is a
>> > > > building
>> > > > block we could use for other features like and HA undercloud.
>> > > > And yes,
>> > > > it does also have a huge impact on developer velocity in that
>> > > > many of
>> > > > us already prefer to use the tool as a means of streamlining
>> > > > our
>> > > > dev/test cycles to minutes instead of hours. Why spend hours
>> > > > running
>> > > > quickstart Ansible scripts when in many cases you can just
>> > > > doit.sh. htt
>> > > > ps://github.com/dprince/undercloud_containers/blob/master/doit.
>> > > > sh
>> > > >
>> > >
>> > > So like I've repeatedly said, I'm not completely against it as I
>> > > agree
>> > > what we have is not ideal.  I'm not -2, I'm -1 pending additional
>> > > information. I'm trying to be realistic and reduce our risk for
>> > > this
>> > > cycle.
>> >
>> >
>> > This reduces our complexity greatly I think in that once it is
>> > completed
>> > will allow 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-04 Thread Dmitry Tantsur

(top-posting, as it is not a direct response to a specific line)

This is your friendly reminder that we're not quite near containerized 
ironic-inspector. The THT for it has probably never been tested at all, and the 
iptables magic we do may simply not be containers-compatible. Milan would 
appreciate any help with his ironic-inspector rework.


Dmitry

On 10/04/2017 03:00 PM, Dan Prince wrote:

On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:

On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
wrote:



On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz 
wrote:


On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince 
wrote:

On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:

Hey Dan,

Thanks for sending out a note about this. I have a few
questions
inline.

On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
wrote:

One of the things the TripleO containers team is planning
on
tackling
in Queens is fully containerizing the undercloud. At the
PTG we
created
an etherpad [1] that contains a list of features that need
to be
implemented to fully replace instack-undercloud.



I know we talked about this at the PTG and I was skeptical
that this
will land in Queens. With the exception of the Container's
team
wanting this, I'm not sure there is an actual end user who is
looking
for the feature so I want to make sure we're not just doing
more work
because we as developers think it's a good idea.


I've heard from several operators that they were actually
surprised we
implemented containers in the Overcloud first. Validating a new
deployment framework on a single node Undercloud (for
operators) before
overtaking their entire cloud deployment has a lot of merit to
it IMO.
When you share the same deployment architecture across the
overcloud/undercloud it puts us in a better position to decide
where to
expose new features to operators first (when creating the
undercloud or
overcloud for example).

Also, if you read my email again I've explicitly listed the
"Containers" benefit last. While I think moving the undercloud
to
containers is a great benefit all by itself this is more of a
"framework alignment" in TripleO and gets us out of maintaining
huge
amounts of technical debt. Re-using the same framework for the
undercloud and overcloud has a lot of merit. It effectively
streamlines
the development process for service developers, and 3rd parties
wishing
to integrate some of their components on a single node. Why be
forced
to create a multi-node dev environment if you don't have to
(aren't
using HA for example).

Lets be honest. While instack-undercloud helped solve the old
"seed" VM
issue it was outdated the day it landed upstream. The entire
premise of
the tool is that it uses old style "elements" to create the
undercloud
and we moved away from those as the primary means driving the
creation
of the Overcloud years ago at this point. The new
'undercloud_deploy'
installer gets us back to our roots by once again sharing the
same
architecture to create the over and underclouds. A demo from
long ago
expands on this idea a bit:  https://www.youtube.com/watch?v=y1
qMDLAf26
Q=5s

In short, we aren't just doing more work because developers
think it is
a good idea. This has potential to be one of the most useful
architectural changes in TripleO that we've made in years.
Could
significantly decrease our CI reasources if we use it to
replace the
existing scenarios jobs which take multiple VMs per job. Is a
building
block we could use for other features like and HA undercloud.
And yes,
it does also have a huge impact on developer velocity in that
many of
us already prefer to use the tool as a means of streamlining
our
dev/test cycles to minutes instead of hours. Why spend hours
running
quickstart Ansible scripts when in many cases you can just
doit.sh. htt
ps://github.com/dprince/undercloud_containers/blob/master/doit.
sh



So like I've repeatedly said, I'm not completely against it as I
agree
what we have is not ideal.  I'm not -2, I'm -1 pending additional
information. I'm trying to be realistic and reduce our risk for
this
cycle.



This reduces our complexity greatly I think in that once it is
completed
will allow us to eliminate two project (instack and instack-
undercloud) and
the maintenance thereof. Furthermore, as this dovetails nice with
the
Ansible



I agree. So I think there's some misconceptions here about my
thoughts
on this effort. I am not against this effort. I am for this effort
and
wish to see more of it. I want to see the effort communicated
publicly
via ML and IRC meetings.  What I am against switching the default
undercloud method until the containerization of the undercloud has
the
appropriate test coverage and documentation to ensure it is on par
with what it is replacing.  Does this make sense?



  IMHO doit.sh is not acceptable as an undercloud installer and
this is what I've been trying to point out as the actual impact
to the
end user 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-04 Thread Dan Prince
On Tue, 2017-10-03 at 16:03 -0600, Alex Schultz wrote:
> On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince 
> wrote:
> > 
> > 
> > On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz 
> > wrote:
> > > 
> > > On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince 
> > > wrote:
> > > > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
> > > > > Hey Dan,
> > > > > 
> > > > > Thanks for sending out a note about this. I have a few
> > > > > questions
> > > > > inline.
> > > > > 
> > > > > On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince  > > > > m>
> > > > > wrote:
> > > > > > One of the things the TripleO containers team is planning
> > > > > > on
> > > > > > tackling
> > > > > > in Queens is fully containerizing the undercloud. At the
> > > > > > PTG we
> > > > > > created
> > > > > > an etherpad [1] that contains a list of features that need
> > > > > > to be
> > > > > > implemented to fully replace instack-undercloud.
> > > > > > 
> > > > > 
> > > > > I know we talked about this at the PTG and I was skeptical
> > > > > that this
> > > > > will land in Queens. With the exception of the Container's
> > > > > team
> > > > > wanting this, I'm not sure there is an actual end user who is
> > > > > looking
> > > > > for the feature so I want to make sure we're not just doing
> > > > > more work
> > > > > because we as developers think it's a good idea.
> > > > 
> > > > I've heard from several operators that they were actually
> > > > surprised we
> > > > implemented containers in the Overcloud first. Validating a new
> > > > deployment framework on a single node Undercloud (for
> > > > operators) before
> > > > overtaking their entire cloud deployment has a lot of merit to
> > > > it IMO.
> > > > When you share the same deployment architecture across the
> > > > overcloud/undercloud it puts us in a better position to decide
> > > > where to
> > > > expose new features to operators first (when creating the
> > > > undercloud or
> > > > overcloud for example).
> > > > 
> > > > Also, if you read my email again I've explicitly listed the
> > > > "Containers" benefit last. While I think moving the undercloud
> > > > to
> > > > containers is a great benefit all by itself this is more of a
> > > > "framework alignment" in TripleO and gets us out of maintaining
> > > > huge
> > > > amounts of technical debt. Re-using the same framework for the
> > > > undercloud and overcloud has a lot of merit. It effectively
> > > > streamlines
> > > > the development process for service developers, and 3rd parties
> > > > wishing
> > > > to integrate some of their components on a single node. Why be
> > > > forced
> > > > to create a multi-node dev environment if you don't have to
> > > > (aren't
> > > > using HA for example).
> > > > 
> > > > Lets be honest. While instack-undercloud helped solve the old
> > > > "seed" VM
> > > > issue it was outdated the day it landed upstream. The entire
> > > > premise of
> > > > the tool is that it uses old style "elements" to create the
> > > > undercloud
> > > > and we moved away from those as the primary means driving the
> > > > creation
> > > > of the Overcloud years ago at this point. The new
> > > > 'undercloud_deploy'
> > > > installer gets us back to our roots by once again sharing the
> > > > same
> > > > architecture to create the over and underclouds. A demo from
> > > > long ago
> > > > expands on this idea a bit:  https://www.youtube.com/watch?v=y1
> > > > qMDLAf26
> > > > Q=5s
> > > > 
> > > > In short, we aren't just doing more work because developers
> > > > think it is
> > > > a good idea. This has potential to be one of the most useful
> > > > architectural changes in TripleO that we've made in years.
> > > > Could
> > > > significantly decrease our CI reasources if we use it to
> > > > replace the
> > > > existing scenarios jobs which take multiple VMs per job. Is a
> > > > building
> > > > block we could use for other features like and HA undercloud.
> > > > And yes,
> > > > it does also have a huge impact on developer velocity in that
> > > > many of
> > > > us already prefer to use the tool as a means of streamlining
> > > > our
> > > > dev/test cycles to minutes instead of hours. Why spend hours
> > > > running
> > > > quickstart Ansible scripts when in many cases you can just
> > > > doit.sh. htt
> > > > ps://github.com/dprince/undercloud_containers/blob/master/doit.
> > > > sh
> > > > 
> > > 
> > > So like I've repeatedly said, I'm not completely against it as I
> > > agree
> > > what we have is not ideal.  I'm not -2, I'm -1 pending additional
> > > information. I'm trying to be realistic and reduce our risk for
> > > this
> > > cycle.
> > 
> > 
> > This reduces our complexity greatly I think in that once it is
> > completed
> > will allow us to eliminate two project (instack and instack-
> > undercloud) and
> > the maintenance thereof. Furthermore, as this dovetails nice with
> > the
> > Ansible
> > 
> 
> I agree. 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Alex Schultz
On Tue, Oct 3, 2017 at 2:46 PM, Dan Prince  wrote:
>
>
> On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz  wrote:
>>
>> On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  wrote:
>> > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
>> >> Hey Dan,
>> >>
>> >> Thanks for sending out a note about this. I have a few questions
>> >> inline.
>> >>
>> >> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
>> >> wrote:
>> >> > One of the things the TripleO containers team is planning on
>> >> > tackling
>> >> > in Queens is fully containerizing the undercloud. At the PTG we
>> >> > created
>> >> > an etherpad [1] that contains a list of features that need to be
>> >> > implemented to fully replace instack-undercloud.
>> >> >
>> >>
>> >> I know we talked about this at the PTG and I was skeptical that this
>> >> will land in Queens. With the exception of the Container's team
>> >> wanting this, I'm not sure there is an actual end user who is looking
>> >> for the feature so I want to make sure we're not just doing more work
>> >> because we as developers think it's a good idea.
>> >
>> > I've heard from several operators that they were actually surprised we
>> > implemented containers in the Overcloud first. Validating a new
>> > deployment framework on a single node Undercloud (for operators) before
>> > overtaking their entire cloud deployment has a lot of merit to it IMO.
>> > When you share the same deployment architecture across the
>> > overcloud/undercloud it puts us in a better position to decide where to
>> > expose new features to operators first (when creating the undercloud or
>> > overcloud for example).
>> >
>> > Also, if you read my email again I've explicitly listed the
>> > "Containers" benefit last. While I think moving the undercloud to
>> > containers is a great benefit all by itself this is more of a
>> > "framework alignment" in TripleO and gets us out of maintaining huge
>> > amounts of technical debt. Re-using the same framework for the
>> > undercloud and overcloud has a lot of merit. It effectively streamlines
>> > the development process for service developers, and 3rd parties wishing
>> > to integrate some of their components on a single node. Why be forced
>> > to create a multi-node dev environment if you don't have to (aren't
>> > using HA for example).
>> >
>> > Lets be honest. While instack-undercloud helped solve the old "seed" VM
>> > issue it was outdated the day it landed upstream. The entire premise of
>> > the tool is that it uses old style "elements" to create the undercloud
>> > and we moved away from those as the primary means driving the creation
>> > of the Overcloud years ago at this point. The new 'undercloud_deploy'
>> > installer gets us back to our roots by once again sharing the same
>> > architecture to create the over and underclouds. A demo from long ago
>> > expands on this idea a bit:  https://www.youtube.com/watch?v=y1qMDLAf26
>> > Q=5s
>> >
>> > In short, we aren't just doing more work because developers think it is
>> > a good idea. This has potential to be one of the most useful
>> > architectural changes in TripleO that we've made in years. Could
>> > significantly decrease our CI reasources if we use it to replace the
>> > existing scenarios jobs which take multiple VMs per job. Is a building
>> > block we could use for other features like and HA undercloud. And yes,
>> > it does also have a huge impact on developer velocity in that many of
>> > us already prefer to use the tool as a means of streamlining our
>> > dev/test cycles to minutes instead of hours. Why spend hours running
>> > quickstart Ansible scripts when in many cases you can just doit.sh. htt
>> > ps://github.com/dprince/undercloud_containers/blob/master/doit.sh
>> >
>>
>> So like I've repeatedly said, I'm not completely against it as I agree
>> what we have is not ideal.  I'm not -2, I'm -1 pending additional
>> information. I'm trying to be realistic and reduce our risk for this
>> cycle.
>
>
> This reduces our complexity greatly I think in that once it is completed
> will allow us to eliminate two project (instack and instack-undercloud) and
> the maintenance thereof. Furthermore, as this dovetails nice with the
> Ansible
>

I agree. So I think there's some misconceptions here about my thoughts
on this effort. I am not against this effort. I am for this effort and
wish to see more of it. I want to see the effort communicated publicly
via ML and IRC meetings.  What I am against switching the default
undercloud method until the containerization of the undercloud has the
appropriate test coverage and documentation to ensure it is on par
with what it is replacing.  Does this make sense?

>>
>>  IMHO doit.sh is not acceptable as an undercloud installer and
>> this is what I've been trying to point out as the actual impact to the
>> end user who has to use this thing.
>
>
> doit.sh is an example of where the effort is today. 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Dan Prince
On Tue, Oct 3, 2017 at 3:50 PM, Alex Schultz  wrote:

> On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  wrote:
> > On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
> >> Hey Dan,
> >>
> >> Thanks for sending out a note about this. I have a few questions
> >> inline.
> >>
> >> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
> >> wrote:
> >> > One of the things the TripleO containers team is planning on
> >> > tackling
> >> > in Queens is fully containerizing the undercloud. At the PTG we
> >> > created
> >> > an etherpad [1] that contains a list of features that need to be
> >> > implemented to fully replace instack-undercloud.
> >> >
> >>
> >> I know we talked about this at the PTG and I was skeptical that this
> >> will land in Queens. With the exception of the Container's team
> >> wanting this, I'm not sure there is an actual end user who is looking
> >> for the feature so I want to make sure we're not just doing more work
> >> because we as developers think it's a good idea.
> >
> > I've heard from several operators that they were actually surprised we
> > implemented containers in the Overcloud first. Validating a new
> > deployment framework on a single node Undercloud (for operators) before
> > overtaking their entire cloud deployment has a lot of merit to it IMO.
> > When you share the same deployment architecture across the
> > overcloud/undercloud it puts us in a better position to decide where to
> > expose new features to operators first (when creating the undercloud or
> > overcloud for example).
> >
> > Also, if you read my email again I've explicitly listed the
> > "Containers" benefit last. While I think moving the undercloud to
> > containers is a great benefit all by itself this is more of a
> > "framework alignment" in TripleO and gets us out of maintaining huge
> > amounts of technical debt. Re-using the same framework for the
> > undercloud and overcloud has a lot of merit. It effectively streamlines
> > the development process for service developers, and 3rd parties wishing
> > to integrate some of their components on a single node. Why be forced
> > to create a multi-node dev environment if you don't have to (aren't
> > using HA for example).
> >
> > Lets be honest. While instack-undercloud helped solve the old "seed" VM
> > issue it was outdated the day it landed upstream. The entire premise of
> > the tool is that it uses old style "elements" to create the undercloud
> > and we moved away from those as the primary means driving the creation
> > of the Overcloud years ago at this point. The new 'undercloud_deploy'
> > installer gets us back to our roots by once again sharing the same
> > architecture to create the over and underclouds. A demo from long ago
> > expands on this idea a bit:  https://www.youtube.com/watch?v=y1qMDLAf26
> > Q=5s
> >
> > In short, we aren't just doing more work because developers think it is
> > a good idea. This has potential to be one of the most useful
> > architectural changes in TripleO that we've made in years. Could
> > significantly decrease our CI reasources if we use it to replace the
> > existing scenarios jobs which take multiple VMs per job. Is a building
> > block we could use for other features like and HA undercloud. And yes,
> > it does also have a huge impact on developer velocity in that many of
> > us already prefer to use the tool as a means of streamlining our
> > dev/test cycles to minutes instead of hours. Why spend hours running
> > quickstart Ansible scripts when in many cases you can just doit.sh. htt
> > ps://github.com/dprince/undercloud_containers/blob/master/doit.sh
> >
>
> So like I've repeatedly said, I'm not completely against it as I agree
> what we have is not ideal.  I'm not -2, I'm -1 pending additional
> information. I'm trying to be realistic and reduce our risk for this
> cycle.


This reduces our complexity greatly I think in that once it is completed
will allow us to eliminate two project (instack and instack-undercloud) and
the maintenance thereof. Furthermore, as this dovetails nice with the
Ansible


>  IMHO doit.sh is not acceptable as an undercloud installer and
> this is what I've been trying to point out as the actual impact to the
> end user who has to use this thing.


doit.sh is an example of where the effort is today. It is essentially the
same stuff we document online here:
http://tripleo.org/install/containers_deployment/undercloud.html.

Similar to quickstart it is just something meant to help you setup a dev
environment.


> We have an established
> installation method for the undercloud, that while isn't great, isn't
> a bash script with git fetches, etc.  So as for the implementation,
> this is what I want to see properly flushed out prior to accepting
> this feature as complete for Queens (and the new default).


Of course the feature would need to prove itself before it becomes the new
default Undercloud. I'm trying to build consensus 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Alex Schultz
On Tue, Oct 3, 2017 at 1:50 PM, Alex Schultz  wrote:
> On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  wrote:
>> On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
>>> Hey Dan,
>>>
>>> Thanks for sending out a note about this. I have a few questions
>>> inline.
>>>
>>> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
>>> wrote:
>>> > One of the things the TripleO containers team is planning on
>>> > tackling
>>> > in Queens is fully containerizing the undercloud. At the PTG we
>>> > created
>>> > an etherpad [1] that contains a list of features that need to be
>>> > implemented to fully replace instack-undercloud.
>>> >
>>>
>>> I know we talked about this at the PTG and I was skeptical that this
>>> will land in Queens. With the exception of the Container's team
>>> wanting this, I'm not sure there is an actual end user who is looking
>>> for the feature so I want to make sure we're not just doing more work
>>> because we as developers think it's a good idea.
>>
>> I've heard from several operators that they were actually surprised we
>> implemented containers in the Overcloud first. Validating a new
>> deployment framework on a single node Undercloud (for operators) before
>> overtaking their entire cloud deployment has a lot of merit to it IMO.
>> When you share the same deployment architecture across the
>> overcloud/undercloud it puts us in a better position to decide where to
>> expose new features to operators first (when creating the undercloud or
>> overcloud for example).
>>
>> Also, if you read my email again I've explicitly listed the
>> "Containers" benefit last. While I think moving the undercloud to
>> containers is a great benefit all by itself this is more of a
>> "framework alignment" in TripleO and gets us out of maintaining huge
>> amounts of technical debt. Re-using the same framework for the
>> undercloud and overcloud has a lot of merit. It effectively streamlines
>> the development process for service developers, and 3rd parties wishing
>> to integrate some of their components on a single node. Why be forced
>> to create a multi-node dev environment if you don't have to (aren't
>> using HA for example).
>>
>> Lets be honest. While instack-undercloud helped solve the old "seed" VM
>> issue it was outdated the day it landed upstream. The entire premise of
>> the tool is that it uses old style "elements" to create the undercloud
>> and we moved away from those as the primary means driving the creation
>> of the Overcloud years ago at this point. The new 'undercloud_deploy'
>> installer gets us back to our roots by once again sharing the same
>> architecture to create the over and underclouds. A demo from long ago
>> expands on this idea a bit:  https://www.youtube.com/watch?v=y1qMDLAf26
>> Q=5s
>>
>> In short, we aren't just doing more work because developers think it is
>> a good idea. This has potential to be one of the most useful
>> architectural changes in TripleO that we've made in years. Could
>> significantly decrease our CI reasources if we use it to replace the
>> existing scenarios jobs which take multiple VMs per job. Is a building
>> block we could use for other features like and HA undercloud. And yes,
>> it does also have a huge impact on developer velocity in that many of
>> us already prefer to use the tool as a means of streamlining our
>> dev/test cycles to minutes instead of hours. Why spend hours running
>> quickstart Ansible scripts when in many cases you can just doit.sh. htt
>> ps://github.com/dprince/undercloud_containers/blob/master/doit.sh
>>
>
> So like I've repeatedly said, I'm not completely against it as I agree
> what we have is not ideal.  I'm not -2, I'm -1 pending additional
> information. I'm trying to be realistic and reduce our risk for this
> cycle.   IMHO doit.sh is not acceptable as an undercloud installer and
> this is what I've been trying to point out as the actual impact to the
> end user who has to use this thing. We have an established
> installation method for the undercloud, that while isn't great, isn't
> a bash script with git fetches, etc.  So as for the implementation,
> this is what I want to see properly flushed out prior to accepting
> this feature as complete for Queens (and the new default).  I would
> like to see a plan of what features need to be added (eg. the stuff on
> the etherpad), folks assigned to do this work, and estimated
> timelines.  Given that we shouldn't be making major feature changes
> after M2 (~9 weeks), I want to get an understanding of what is
> realistically going to make it.  If after reviewing the initial
> details we find that it's not actually going to make M2, then let's
> agree to this now rather than trying to force it in at the end.
>
> I know you've been a great proponent of the containerized undercloud
> and I agree it offers a lot more for development efforts. But I just
> want to make sure that we are getting all the feedback we can 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Alex Schultz
On Tue, Oct 3, 2017 at 11:12 AM, Dan Prince  wrote:
> On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
>> Hey Dan,
>>
>> Thanks for sending out a note about this. I have a few questions
>> inline.
>>
>> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
>> wrote:
>> > One of the things the TripleO containers team is planning on
>> > tackling
>> > in Queens is fully containerizing the undercloud. At the PTG we
>> > created
>> > an etherpad [1] that contains a list of features that need to be
>> > implemented to fully replace instack-undercloud.
>> >
>>
>> I know we talked about this at the PTG and I was skeptical that this
>> will land in Queens. With the exception of the Container's team
>> wanting this, I'm not sure there is an actual end user who is looking
>> for the feature so I want to make sure we're not just doing more work
>> because we as developers think it's a good idea.
>
> I've heard from several operators that they were actually surprised we
> implemented containers in the Overcloud first. Validating a new
> deployment framework on a single node Undercloud (for operators) before
> overtaking their entire cloud deployment has a lot of merit to it IMO.
> When you share the same deployment architecture across the
> overcloud/undercloud it puts us in a better position to decide where to
> expose new features to operators first (when creating the undercloud or
> overcloud for example).
>
> Also, if you read my email again I've explicitly listed the
> "Containers" benefit last. While I think moving the undercloud to
> containers is a great benefit all by itself this is more of a
> "framework alignment" in TripleO and gets us out of maintaining huge
> amounts of technical debt. Re-using the same framework for the
> undercloud and overcloud has a lot of merit. It effectively streamlines
> the development process for service developers, and 3rd parties wishing
> to integrate some of their components on a single node. Why be forced
> to create a multi-node dev environment if you don't have to (aren't
> using HA for example).
>
> Lets be honest. While instack-undercloud helped solve the old "seed" VM
> issue it was outdated the day it landed upstream. The entire premise of
> the tool is that it uses old style "elements" to create the undercloud
> and we moved away from those as the primary means driving the creation
> of the Overcloud years ago at this point. The new 'undercloud_deploy'
> installer gets us back to our roots by once again sharing the same
> architecture to create the over and underclouds. A demo from long ago
> expands on this idea a bit:  https://www.youtube.com/watch?v=y1qMDLAf26
> Q=5s
>
> In short, we aren't just doing more work because developers think it is
> a good idea. This has potential to be one of the most useful
> architectural changes in TripleO that we've made in years. Could
> significantly decrease our CI reasources if we use it to replace the
> existing scenarios jobs which take multiple VMs per job. Is a building
> block we could use for other features like and HA undercloud. And yes,
> it does also have a huge impact on developer velocity in that many of
> us already prefer to use the tool as a means of streamlining our
> dev/test cycles to minutes instead of hours. Why spend hours running
> quickstart Ansible scripts when in many cases you can just doit.sh. htt
> ps://github.com/dprince/undercloud_containers/blob/master/doit.sh
>

So like I've repeatedly said, I'm not completely against it as I agree
what we have is not ideal.  I'm not -2, I'm -1 pending additional
information. I'm trying to be realistic and reduce our risk for this
cycle.   IMHO doit.sh is not acceptable as an undercloud installer and
this is what I've been trying to point out as the actual impact to the
end user who has to use this thing. We have an established
installation method for the undercloud, that while isn't great, isn't
a bash script with git fetches, etc.  So as for the implementation,
this is what I want to see properly flushed out prior to accepting
this feature as complete for Queens (and the new default).  I would
like to see a plan of what features need to be added (eg. the stuff on
the etherpad), folks assigned to do this work, and estimated
timelines.  Given that we shouldn't be making major feature changes
after M2 (~9 weeks), I want to get an understanding of what is
realistically going to make it.  If after reviewing the initial
details we find that it's not actually going to make M2, then let's
agree to this now rather than trying to force it in at the end.

I know you've been a great proponent of the containerized undercloud
and I agree it offers a lot more for development efforts. But I just
want to make sure that we are getting all the feedback we can before
continuing down this path.  Since, as you point out, a bunch of this
work is already available for consumption by developers, I don't see
making it the new default as a 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Emilien Macchi
On Tue, Oct 3, 2017 at 10:12 AM, Dan Prince  wrote:
[...]
> I would let other chime in but the feedback I've gotten has mostly been
>  that it improves the dev/test cycle greatly.

[...]

I like both aschultz & dprince thoughts here, I agree with both of you
on most of the points made here.
I think we need to engage more efforts on CI (see what Alex wrote
about milestone, we should try to respect that, it has proved to be
helpful) & documentation as well (let's push doc before end of m2).

If we can make CI & doc working before end of m2, it's at least a good
step forward.
Also, we could ship this feature as "experimental" in Queens and
"stable" in Rocky (even if it has been developed since more than one
cycle by dprince & co), I think it's a reasonable path for our users.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-03 Thread Dan Prince
On Mon, 2017-10-02 at 15:20 -0600, Alex Schultz wrote:
> Hey Dan,
> 
> Thanks for sending out a note about this. I have a few questions
> inline.
> 
> On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince 
> wrote:
> > One of the things the TripleO containers team is planning on
> > tackling
> > in Queens is fully containerizing the undercloud. At the PTG we
> > created
> > an etherpad [1] that contains a list of features that need to be
> > implemented to fully replace instack-undercloud.
> > 
> 
> I know we talked about this at the PTG and I was skeptical that this
> will land in Queens. With the exception of the Container's team
> wanting this, I'm not sure there is an actual end user who is looking
> for the feature so I want to make sure we're not just doing more work
> because we as developers think it's a good idea.

I've heard from several operators that they were actually surprised we
implemented containers in the Overcloud first. Validating a new
deployment framework on a single node Undercloud (for operators) before
overtaking their entire cloud deployment has a lot of merit to it IMO.
When you share the same deployment architecture across the
overcloud/undercloud it puts us in a better position to decide where to
expose new features to operators first (when creating the undercloud or
overcloud for example).

Also, if you read my email again I've explicitly listed the
"Containers" benefit last. While I think moving the undercloud to
containers is a great benefit all by itself this is more of a
"framework alignment" in TripleO and gets us out of maintaining huge
amounts of technical debt. Re-using the same framework for the
undercloud and overcloud has a lot of merit. It effectively streamlines
the development process for service developers, and 3rd parties wishing
to integrate some of their components on a single node. Why be forced
to create a multi-node dev environment if you don't have to (aren't
using HA for example).

Lets be honest. While instack-undercloud helped solve the old "seed" VM
issue it was outdated the day it landed upstream. The entire premise of
the tool is that it uses old style "elements" to create the undercloud
and we moved away from those as the primary means driving the creation
of the Overcloud years ago at this point. The new 'undercloud_deploy'
installer gets us back to our roots by once again sharing the same
architecture to create the over and underclouds. A demo from long ago
expands on this idea a bit:  https://www.youtube.com/watch?v=y1qMDLAf26
Q=5s

In short, we aren't just doing more work because developers think it is
a good idea. This has potential to be one of the most useful
architectural changes in TripleO that we've made in years. Could
significantly decrease our CI reasources if we use it to replace the
existing scenarios jobs which take multiple VMs per job. Is a building
block we could use for other features like and HA undercloud. And yes,
it does also have a huge impact on developer velocity in that many of
us already prefer to use the tool as a means of streamlining our
dev/test cycles to minutes instead of hours. Why spend hours running
quickstart Ansible scripts when in many cases you can just doit.sh. htt
ps://github.com/dprince/undercloud_containers/blob/master/doit.sh

Lastly, this isn't just a containers team thing. We've been using the
undercloud_deploy architecture across many teams to help develop for
almost an entire cycle now. Huge benefits. I would go as far as saying
that undercloud_deploy was *the* biggest feature in Pike that enabled
us to bang out a majority of the docker/service templates in tripleo-
heat-templates.

>  Given that etherpad
> appears to contain a pretty big list of features, are we going to be
> able to land all of them by M2?  Would it be beneficial to craft a
> basic spec related to this to ensure we are not missing additional
> things?

I'm not sure there is a lot of value in creating a spec at this point.
We've already got an approved blueprint for the feature in Pike here: h
ttps://blueprints.launchpad.net/tripleo/+spec/containerized-undercloud

I think we might get more velocity out of grooming the etherpad and
perhaps dividing this work among the appropriate teams.

> 
> > Benefits of this work:
> > 
> >  -Alignment: aligning the undercloud and overcloud installers gets
> > rid
> > of dual maintenance of services.
> > 
> 
> I like reusing existing stuff. +1
> 
> >  -Composability: tripleo-heat-templates and our new Ansible
> > architecture around it are composable. This means any set of
> > services
> > can be used to build up your own undercloud. In other words the
> > framework here isn't just useful for "underclouds". It is really
> > the
> > ability to deploy Tripleo on a single node with no external
> > dependencies. Single node TripleO installer. The containers team
> > has
> > already been leveraging existing (experimental) undercloud_deploy
> > installer to develop services for Pike.
> > 
> 

Re: [openstack-dev] [TripleO] containerized undercloud in Queens

2017-10-02 Thread Alex Schultz
Hey Dan,

Thanks for sending out a note about this. I have a few questions inline.

On Mon, Oct 2, 2017 at 6:02 AM, Dan Prince  wrote:
> One of the things the TripleO containers team is planning on tackling
> in Queens is fully containerizing the undercloud. At the PTG we created
> an etherpad [1] that contains a list of features that need to be
> implemented to fully replace instack-undercloud.
>

I know we talked about this at the PTG and I was skeptical that this
will land in Queens. With the exception of the Container's team
wanting this, I'm not sure there is an actual end user who is looking
for the feature so I want to make sure we're not just doing more work
because we as developers think it's a good idea. Given that etherpad
appears to contain a pretty big list of features, are we going to be
able to land all of them by M2?  Would it be beneficial to craft a
basic spec related to this to ensure we are not missing additional
things?

> Benefits of this work:
>
>  -Alignment: aligning the undercloud and overcloud installers gets rid
> of dual maintenance of services.
>

I like reusing existing stuff. +1

>  -Composability: tripleo-heat-templates and our new Ansible
> architecture around it are composable. This means any set of services
> can be used to build up your own undercloud. In other words the
> framework here isn't just useful for "underclouds". It is really the
> ability to deploy Tripleo on a single node with no external
> dependencies. Single node TripleO installer. The containers team has
> already been leveraging existing (experimental) undercloud_deploy
> installer to develop services for Pike.
>

Is this something that is actually being asked for or is this just an
added bonus because it allows developers to reduce what is actually
being deployed for testing?

>  -Development: The containerized undercloud is a great development
> tool. It utilizes the same framework as the full overcloud deployment
> but takes about 20 minutes to deploy.  This means faster iterations,
> less waiting, and more testing.  Having this be a first class citizen
> in the ecosystem will ensure this platform is functioning for
> developers to use all the time.
>

Seems to go with the previous question about the re-usability for
people who are not developers.  Has everyone (including non-container
folks) tried this out and attest that it's a better workflow for them?
 Are there use cases that are made worse by switching?

>  -CI resources: better use of CI resources. At the PTG we received
> feedback from the OpenStack infrastructure team that our upstream CI
> resource usage is quite high at times (even as high as 50% of the
> total). Because of the shared framework and single node capabilities we
> can re-architecture much of our upstream CI matrix around single node.
> We no longer require multinode jobs to be able to test many of the
> services in tripleo-heat-templates... we can just use a single cloud VM
> instead. We'll still want multinode undercloud -> overcloud jobs for
> testing things like HA and baremetal provisioning. But we can cover a
> large set of the services (in particular many of the new scenario jobs
> we added in Pike) with single node CI test runs in much less time.
>

I like this idea but would like to see more details around this.
Since this is a new feature we need to make sure that we are properly
covering the containerized undercloud with CI as well.  I think we
need 3 jobs to properly cover this feature before marking it done. I
added them to the etherpad but I think we need to ensure the following
3 jobs are defined and voting by M2 to consider actually switching
from the current instack-undercloud installation to the containerized
version.

1) undercloud-containers - a containerized install, should be voting by m1
2) undercloud-containers-update - minor updates run on containerized
underclouds, should be voting by m2
3) undercloud-containers-upgrade - major upgrade from
non-containerized to containerized undercloud, should be voting by m2.

If we have these jobs, is there anything we can drop or mark as
covered that is currently being covered by an overcloud job?

>  -Containers: There are no plans to containerize the existing instack-
> undercloud work. By moving our undercloud installer to a tripleo-heat-
> templates and Ansible architecture we can leverage containers.
> Interestingly, the same installer also supports baremetal (package)
> installation as well at this point. Like to overcloud however I think
> making containers our undercloud default would better align the TripleO
> tooling.
>
> We are actively working through a few issues with the deployment
> framework Ansible effort to fully integrate that into the undercloud
> installer. We are also reaching out to other teams like the UI and
> Security folks to coordinate the efforts around those components. If
> there are any questions about the effort or you'd like to be involved
> in the