Re: [openstack-dev] [tripleo] Roadmap for Container CI work
On 6.4.2017 12:46, Jiří Stránský wrote: On 4.4.2017 22:01, Emilien Macchi wrote: After our weekly meeting of today, I found useful to share and discuss our roadmap for Container CI jobs in TripleO. They are ordered by priority from the highest to lowest: 1. Swap ovb-nonha job with ovb-containers, enable introspection on the container job and shuffle other coverage (e.g ssl) to other jobs (HA?). It will help us to get coverage for ovb-containers scenario again, without consuming more rh1 resources and keep existing coverage. 2. Get multinode coverage of deployments - this should integrate with the scenarios we already have defined for non-container deployment. This is super important to cover all overcloud services, like we did with classic deployments. It should be non voting to start and then voting once it works. We should find a way to keep the same templates as we have now, and just include the docker environment. In other words, find a way to keep using: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode.yaml so we don't duplicate scenario environments. 3. Implement container upgrade job, which for Pike will be deploy a baremetal overcloud, then migrate on upgrade to containers. Use multinode jobs for this task. Start with a non-voting job and move to the gate once it work. I also suggest to use scenarios framework, so we keep good coverage. The first iteration of this job is ready to be reviewed and landed. Please see the patches here [1]. The latest job execution didn't go all the way to success yet, it failed during Ansible upgrade steps execution [2], but i think the patches are now far enough that they would be good to merge anyway, and issues can be ironed out subsequently, as well as making the job actually Ocata->master rather than master->master (currently just switching from non-containers to containers). [1] https://review.openstack.org/#/q/topic:container-upgrade [2] http://logs.openstack.org/84/450784/8/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/a1850f7/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz Sorry the [2] link was incorrect, this is the right one: http://logs.openstack.org/84/450784/9/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/23f9190/logs/undercloud/home/jenkins/overcloud_upgrade_console.log.txt.gz 4. After we implement the workflow for minor updates, have a job with tests container-to-container updates for minor (rolling) updates, this ideally should add some coverage to ensure no downtime of APIs and possibly checks for service restarts (ref recent bugs about bouncing services on minor updates) 5. Once Pike is released and Queens starts, let's work on container to containers upgrade job. Any feedback or question is highly welcome, Note: The proposal comes from shardy's notes on https://etherpad.openstack.org/p/tripleo-container-ci - feel free to contribute to the etherpad or mailing list. Thanks, __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Roadmap for Container CI work
On 4.4.2017 22:01, Emilien Macchi wrote: After our weekly meeting of today, I found useful to share and discuss our roadmap for Container CI jobs in TripleO. They are ordered by priority from the highest to lowest: 1. Swap ovb-nonha job with ovb-containers, enable introspection on the container job and shuffle other coverage (e.g ssl) to other jobs (HA?). It will help us to get coverage for ovb-containers scenario again, without consuming more rh1 resources and keep existing coverage. 2. Get multinode coverage of deployments - this should integrate with the scenarios we already have defined for non-container deployment. This is super important to cover all overcloud services, like we did with classic deployments. It should be non voting to start and then voting once it works. We should find a way to keep the same templates as we have now, and just include the docker environment. In other words, find a way to keep using: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode.yaml so we don't duplicate scenario environments. 3. Implement container upgrade job, which for Pike will be deploy a baremetal overcloud, then migrate on upgrade to containers. Use multinode jobs for this task. Start with a non-voting job and move to the gate once it work. I also suggest to use scenarios framework, so we keep good coverage. The first iteration of this job is ready to be reviewed and landed. Please see the patches here [1]. The latest job execution didn't go all the way to success yet, it failed during Ansible upgrade steps execution [2], but i think the patches are now far enough that they would be good to merge anyway, and issues can be ironed out subsequently, as well as making the job actually Ocata->master rather than master->master (currently just switching from non-containers to containers). [1] https://review.openstack.org/#/q/topic:container-upgrade [2] http://logs.openstack.org/84/450784/8/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/a1850f7/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz 4. After we implement the workflow for minor updates, have a job with tests container-to-container updates for minor (rolling) updates, this ideally should add some coverage to ensure no downtime of APIs and possibly checks for service restarts (ref recent bugs about bouncing services on minor updates) 5. Once Pike is released and Queens starts, let's work on container to containers upgrade job. Any feedback or question is highly welcome, Note: The proposal comes from shardy's notes on https://etherpad.openstack.org/p/tripleo-container-ci - feel free to contribute to the etherpad or mailing list. Thanks, __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Roadmap for Container CI work
On Tue, Apr 04, 2017 at 04:01:48PM -0400, Emilien Macchi wrote: > After our weekly meeting of today, I found useful to share and discuss > our roadmap for Container CI jobs in TripleO. > They are ordered by priority from the highest to lowest: > > 1. Swap ovb-nonha job with ovb-containers, enable introspection on the > container job and shuffle other coverage (e.g ssl) to other jobs > (HA?). It will help us to get coverage for ovb-containers scenario > again, without consuming more rh1 resources and keep existing > coverage. > 2. Get multinode coverage of deployments - this should integrate with > the scenarios we already have defined for non-container deployment. > This is super important to cover all overcloud services, like we did > with classic deployments. It should be non voting to start and then > voting once it works. We should find a way to keep the same templates > as we have now, and just include the docker environment. In other > words, find a way to keep using: > https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode.yaml > so we don't duplicate scenario environments. > 3. Implement container upgrade job, which for Pike will be deploy a > baremetal overcloud, then migrate on upgrade to containers. Use > multinode jobs for this task. Start with a non-voting job and move to > the gate once it work. I also suggest to use scenarios framework, so > we keep good coverage. > 4. After we implement the workflow for minor updates, have a job with > tests container-to-container updates for minor (rolling) updates, this > ideally should add some coverage to ensure no downtime of APIs and > possibly checks for service restarts (ref recent bugs about bouncing > services on minor updates) > 5. Once Pike is released and Queens starts, let's work on container to > containers upgrade job. > > Any feedback or question is highly welcome, +1, I think this makes sense and is well aligned with what we discussed in the meeting. I agree the priority is roughly in the order listed above, but provided we have sufficient folks willing to help we can probably work on some of these tasks in parallel, as really we need at least (1), (2) and (3) ASAP. I've started looking at (4) but there is significant work required to enable this as our current breakpoint based update workflow won't work, and it looks like we also can't use the rolling update feature of SoftwareDeploymentGroup, because we want each node to be fully updated before moving on to the next. Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Roadmap for Container CI work
On Tue, 2017-04-04 at 16:01 -0400, Emilien Macchi wrote: > After our weekly meeting of today, I found useful to share and > discuss > our roadmap for Container CI jobs in TripleO. > They are ordered by priority from the highest to lowest: > > 1. Swap ovb-nonha job with ovb-containers, enable introspection on > the > container job and shuffle other coverage (e.g ssl) to other jobs > (HA?). It will help us to get coverage for ovb-containers scenario > again, without consuming more rh1 resources and keep existing > coverage. The existing containers job already had introspection enabled I think. So this is largely a swap (containers for nonha). We may move some of the SSL features into the existing HA job if that makes more sense to keep coverage on par with what the nonha job was already covering. > 2. Get multinode coverage of deployments - this should integrate with > the scenarios we already have defined for non-container deployment. > This is super important to cover all overcloud services, like we did > with classic deployments. It should be non voting to start and then > voting once it works. We should find a way to keep the same templates > as we have now, and just include the docker environment. In other > words, find a way to keep using: > https://github.com/openstack/tripleo-heat-templates/blob/master/ci/en > vironments/scenario001-multinode.yaml > so we don't duplicate scenario environments. > 3. Implement container upgrade job, which for Pike will be deploy a > baremetal overcloud, then migrate on upgrade to containers. Use > multinode jobs for this task. Start with a non-voting job and move to > the gate once it work. I also suggest to use scenarios framework, so > we keep good coverage. Using multinode here is a great start. We might also at some point decide to swap out the existing OVB job to be a baremetal -> containers version so we have end to end coverage there as well. > 4. After we implement the workflow for minor updates, have a job with > tests container-to-container updates for minor (rolling) updates, > this > ideally should add some coverage to ensure no downtime of APIs and > possibly checks for service restarts (ref recent bugs about bouncing > services on minor updates) > 5. Once Pike is released and Queens starts, let's work on container > to > containers upgrade job. > > Any feedback or question is highly welcome, > > Note: The proposal comes from shardy's notes on > https://etherpad.openstack.org/p/tripleo-container-ci - feel free to > contribute to the etherpad or mailing list. > > Thanks, __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Roadmap for Container CI work
After our weekly meeting of today, I found useful to share and discuss our roadmap for Container CI jobs in TripleO. They are ordered by priority from the highest to lowest: 1. Swap ovb-nonha job with ovb-containers, enable introspection on the container job and shuffle other coverage (e.g ssl) to other jobs (HA?). It will help us to get coverage for ovb-containers scenario again, without consuming more rh1 resources and keep existing coverage. 2. Get multinode coverage of deployments - this should integrate with the scenarios we already have defined for non-container deployment. This is super important to cover all overcloud services, like we did with classic deployments. It should be non voting to start and then voting once it works. We should find a way to keep the same templates as we have now, and just include the docker environment. In other words, find a way to keep using: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode.yaml so we don't duplicate scenario environments. 3. Implement container upgrade job, which for Pike will be deploy a baremetal overcloud, then migrate on upgrade to containers. Use multinode jobs for this task. Start with a non-voting job and move to the gate once it work. I also suggest to use scenarios framework, so we keep good coverage. 4. After we implement the workflow for minor updates, have a job with tests container-to-container updates for minor (rolling) updates, this ideally should add some coverage to ensure no downtime of APIs and possibly checks for service restarts (ref recent bugs about bouncing services on minor updates) 5. Once Pike is released and Queens starts, let's work on container to containers upgrade job. Any feedback or question is highly welcome, Note: The proposal comes from shardy's notes on https://etherpad.openstack.org/p/tripleo-container-ci - feel free to contribute to the etherpad or mailing list. Thanks, -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
