On Wed, Nov 8, 2017 at 5:10 PM, Wesley Hayutin <whayu...@redhat.com> wrote:
> > > On Wed, Nov 8, 2017 at 5:00 PM, Alex Schultz <aschu...@redhat.com> wrote: > >> On Tue, Nov 7, 2017 at 2:59 PM, Emilien Macchi <emil...@redhat.com> >> wrote: >> > On Wed, Nov 8, 2017 at 3:30 AM, James Slagle <james.sla...@gmail.com> >> wrote: >> >> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi <emil...@redhat.com> >> wrote: >> >>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince <dpri...@redhat.com> >> wrote: >> >>> [...] >> >>> >> >>>> -CI resources: better use of CI resources. At the PTG we received >> >>>> feedback from the OpenStack infrastructure team that our upstream CI >> >>>> resource usage is quite high at times (even as high as 50% of the >> >>>> total). Because of the shared framework and single node capabilities >> we >> >>>> can re-architecture much of our upstream CI matrix around single >> node. >> >>>> We no longer require multinode jobs to be able to test many of the >> >>>> services in tripleo-heat-templates... we can just use a single cloud >> VM >> >>>> instead. We'll still want multinode undercloud -> overcloud jobs for >> >>>> testing things like HA and baremetal provisioning. But we can cover a >> >>>> large set of the services (in particular many of the new scenario >> jobs >> >>>> we added in Pike) with single node CI test runs in much less time. >> >>> >> >>> After the last (terrible) weeks in CI, it's pretty clear we need to >> >>> find a solution to reduce and optimize our testing. >> >>> I'm now really convinced by switching our current scenarios jobs to >> >>> NOT deploy the overcloud, and just an undercloud with composable >> >>> services & run tempest. >> >> >> >> +1 if you mean just the scenarios. >> > >> > Yes, just scenarios. >> > >> >> I think we need to keep at least 1 multinode job voting that deploys >> >> the overcloud, probably containers-multinode. >> > >> > Yes, exactly, and also work on optimizing OVB jobs (maybe just keep >> > one or 2 jobs, instead 3). >> > >> >>> Benefits: >> >>> - deploy 1 node instead of 2 nodes, so we save nodepool resources >> >>> - faster (no overcloud) >> >>> - reduce gate queue time, faster development process, faster CI >> >>> >> >>> Challenges: >> >>> - keep overcloud testing, with OVB >> >> >> >> This is why I'm not sure what you're proposing. Do you mean switch all >> >> multinode jobs to be just an undercloud, or just the scenarios? >> > >> > Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be >> > tested with multinode though but well). >> > >> >>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic >> >>> containerized services on overcloud. >> >>> >> >>> I really want to get consensus on these points, please raise your >> >>> voice now before we engage some work on that front. >> >> >> >> I'm fine to optimize the scenarios to be undercloud driven, but feel >> >> we still need a multinode job that deploys the overcloud in the gate. >> >> Otherwise, we'll have nothing that deploys an overcloud in the gate, >> >> which is a step in the wrong direction imo. Primarily, b/c of the loss >> >> of coverage around mistral and all of our workflows. Perhaps down the >> >> road we could find ways to optimize that by using an ephemeral Mistral >> >> (similar to the ephemeral Heat container), and then use a single node, >> >> but we're not there yet. >> >> >> >> On the other hand, if the goal is just to test less upstream so that >> >> we can more quickly merge code, then *not* deploying an overcloud in >> >> the gate at all seems to fit that goal. Is that what you're after? >> > >> > Yes. Thanks for reformulate with better words. >> > Just to be clear, I want to transform the scenarios into single-node >> > jobs that deploy the SAME services (using composable services) from >> > the undercloud, using the new ansible installer. I also want to keep >> > running Tempest. >> > And of course, like we said, keep one multinode job to test overcloud >> > workflow, and OVB with some adjustments. >> > >> >> So I'm ok with switching to use the containerized undercloud deploy to >> smoke test functionality of more complex openstack service >> deployments. What I would like to see prior to investing in this is >> that the plain containerized undercloud deploy job reliability is on >> par with the existing undercloud install. We had to switch the >> undercloud-containers back to non-voting due to higher failure rates >> and it is still not voting. > > > Agree, once we have a little success I'll update featureset027 ( the > undercloud-containers ) > which is still non-voting to use this updated containerized deployment. > Then we'll compare > the undercloud-oooq to undercloud-containers (fs027) After a few weeks of > running. > > >> With the current state of CI being >> questionable due to random failures which are not fully have resolved, >> I would prefer that we ensure existing CI is stable and that what we >> plan to move is as stable. >> > > Agreed, > There are times IMHO when one must strike when the iron is hot on certain > parts of the work here. I felt compelled to help bootstrap Ian with the > containerized undercloud work or see old habits remain and have things > running in tripleo.sh or other shell scripts. > > The priority is to help stablize the upstream CI for sure! > > >> >> Additionally I think we need to ensure that the ovb jobs that still do >> full deployment process become voting by switching to 3rd party CI. >> > > Yes, agreed. I'll reach out to the rdo sf admins and see if we get it > voting -2. > That is what we discussed at summit. > > >> >> Thanks, >> -Alex >> >> > Is it good? >> > >> > Thanks, >> > -- >> > Emilien Macchi >> > >> > ____________________________________________________________ >> ______________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: openstack-dev-requ...@lists.op >> enstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > FYI.. I thought it would be a good idea to allow folks to work on the containerized undercloud w/o using a bunch of upstream resources. https://review.rdoproject.org/r/#/c/10468/ The above patch would allow us to recheck the containerized undercloud install w/o running recheck. You would need a change on instack-undercloud and to execute "check rdo experimental" Reviews are welcome Thanks
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev