ok :) I think a decision has been made so we'll push forward and once everything in moved to rhel8 we will start flooding the issues and try to get it all fixed as quickly as possible.
Thanks everyone Dafna On Thu, Feb 13, 2020 at 10:18 AM Yedidyah Bar David <[email protected]> wrote: > On Thu, Feb 13, 2020 at 12:09 PM Dafna Ron <[email protected]> wrote: > > > > > > > > On Wed, Feb 12, 2020 at 11:37 PM Michal Skrivanek < > [email protected]> wrote: > >> > >> > >> > >> On 12 Feb 2020, at 13:48, Dafna Ron <[email protected]> wrote: > >> > >> > >> > >> On Wed, Feb 12, 2020 at 6:38 PM Martin Perina <[email protected]> > wrote: > >>> > >>> > >>> > >>> On Wed, Feb 12, 2020 at 6:33 PM Dafna Ron <[email protected]> wrote: > >>>> > >>>> > >>>> > >>>> On Wed, Feb 12, 2020 at 5:03 PM Martin Perina <[email protected]> > wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I'm probably missing something here, so please correct me if I'm > wrong: > >>>>> > >>>>> 1. CQ is running OST to detect failures and if OST will not pass, > RPM with the change will not land in tested repo, right? > >>>> > >>>> unless someone manually moves packages into the tested repo - which > caused all projects to fail on engine deploy. > >>>>> > >>>>> > >>>>> 2. OST is still running engine on EL7, right? > >>>> > >>>> yes > >>>>> > >>>>> > >>>>> 3. We have dropped EL7 builds from engine on Feb 6th > https://gerrit.ovirt.org/106795 > >>>>> So no matter what we push to engine repo, it will not pass the > CQ, because it will still be testing the old broken version (because new > builds will produce packages only for FC30 and EL8 repos). Am I right or > missing something? > >>>>> > >>>> Actually, engine has been failing for more then 4 weeks already on > unrelated issues. currently its failing on a package which is deprecated > because the change has not passed CQ and the package that is tested is > based on some un-merged change. so fixing the issues now, have nothing to > do with the missing el8 packages and the longer you wait to fix the > failures, the more regressions you will introduce and harder it will be to > fix it. > >>>> > >>>> At this point, I need to have patch > https://gerrit.ovirt.org/#/c/106809/ run in CQ so I can see why its > failing. > >>> > >>> > >>> Well, there are other fixes after this patch which fixes known issues, > so I'm not sure if only this fix will make the OST to run successfully. > >>> > >>>> > >>>>> Right now we are doing our best to make engine running on CentOS 8 > (my personal estimation is Friday), so after that we should be able to > merge OST patch https://gerrit.ovirt.org/106824 so OST will use EL8 for > engine. > >>>>> > >>>>> > >>>>> So unless I'm missing something it doesn't make sense to block > merging to engine and its dependencies, because we need to make them > running on EL8 and switch engine on EL8 in OST to unblock CQ. Am I right or > am I missing something? > >>>> > >>>> > >>>> lets first get to a point where we are blocked by rl8 related issues > instead of merging more changes which we later on cannot track. > >>>> Lets see what the change is failing as for now, you are blocking all > other projects (not just engine and its deps). > >>> > >>> > >>> Well maybe, but again I don't believe that we are able to stabilize > OST before we get to engine EL8, because since last week there were merged > several patches which might have help OST but also several fixes which > broke engine on EL7. And all those fixes are mixed. > >>> > >>> So from my point right now the biggest priority is to make engine > running on EL8 asap and only afterwards focus on OST stabilization and > fixing other less important dependencies > >> > >> > >> However we should (engine maintainers should) be cautious about merging > other stuff unrelated to el8. If it’s not essential I would suggest to > postpone merging exactly for that reason that we don’t have any validation > beyond unit tests. > >> > >> > >> OST is not related at all to this and its not about stabilizing OST but > about stabilizing engine as these are code issues and not random OST > failures. > >> > >> > >> It’s related in a sense that as soon as we get OST on el8 running we > can sleep a bit better when merging engine patches. > >> > >> The failures in cq are engine related and the rest of the projects are > also failing because a faulty engine package has been introduced to the > rest of the projects so everyone are merging "blind" because of the current > failures. > >> > >> > >> You’re right about the other projects, but we can as well just go back > to a previous build then. Just remove the last ovirt-engine manually? > > > > > > We do not have a roll back on the repo but perhaps Anton would have an > idea. > > I do not see much point. This will give you a somewhat older el7 > build. Newer ones do not exist anymore, because we do not build for > el7 anymore. > > I agree that the best plan is to push strongly for moving to el8. > > I talked with Galit about this earlier today. She is working on a > patch to OST to move the engine there to el8. As we agreed in private, > there is no point in checking it against the current tested repo, > because it's broken. Instead, it should be checked against the current > engine master. If it passes, I think we can merge this patch to OST, > and then hopefully CI/change-queue will be in a good-enough shape to > at least help find real bugs and fix them :-) > > > > >> > >> you can continue merging if you like, but note that all the projects in > CQ have no testing coverage at all at this time and you are merging without > any CI testing. > >> > >> > >> > >>> > >>> Regards, > >>> Martin > >>> > >>> > >>> On Wed, Feb 12, 2020 at 4:03 PM Dafna Ron <[email protected]> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> Engine has been failing for a while and someone has also moved the > broken package manually to the tested repo which broke all projects in cq. > >>>> At this time, until we stabilise engine and get a new package into > tested, all projects will continue failing cq and no new packages will be > moved to the stable repo (tested). > >>>> if you continue to merge, we not only risk adding more regression but > it also makes it more difficult to debug and fix as there are a lot of > changes in the queue waiting to be tested and very little resources to run > it. > >>>> > >>>> Please stop merging on all projects until this is resolved. > >>>> > >>>> Thanks, > >>>> Dafna > >>>> > >>>> > >>>> > >>> > >>> > >>> -- > >>> Martin Perina > >>> Manager, Software Engineering > >>> Red Hat Czech s.r.o. > >>> > >>> > >>> > >>> -- > >>> Martin Perina > >>> Manager, Software Engineering > >>> Red Hat Czech s.r.o. > >> > >> _______________________________________________ > >> Devel mailing list -- [email protected] > >> To unsubscribe send an email to [email protected] > >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >> oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > >> List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/OJXQXGU4KG6WLFKNK2UAYPYXK4UQTGR6/ > > > > _______________________________________________ > > Devel mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/BD2UZXENQME7SUO3H55N5DJULXQKZ7QQ/ > > > > -- > Didi > >
_______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/ZBNUL45CLXKBKO4HVQ5SO6K64PDJYBWN/
