What happened with the SpringBoot. I made the change and actually thought the PR reviews would alert the right people and you could not get a PR in without approval from the devs responsible for that area. This is just since I don't know who is working on what and on what level. I don't know how the wording in this rule could be set so that the right person has actually been contacted.
One other thing to take note of is that this policy applies for all the broken tests, not just flaky ones. So if you manage to get a bad commit in, it is then ok to ignore all the tests that that commit broke and now the responsibility of fixing the codes are on the person that wrote the test that picks up the bugs or changes someone else introduced. I know I like to over think these regulations, but due to the structure at Apache we agree on rules and some can be taken advantage of. If the thing mentioned above happens, there is not one great leader who can give an order to fix the codes. Also if a community member gets a commit in, there is no handle for us to get them to fix anything. It is the community's problem after that. I have to give this a -1, this has to target just flaky tests. If a commit breaks the test, the right thing would be to revert ASAP. If we really need to get something in that breaks tests in another location, and those tests need to be redone, then we should vote for that change to go in as an exception and ignore the broken tests in the same commit bundle. Acknowledging there is no guarantee the tests commented out will ever be fixed, even if they show a bug was introduced. So for example in the current case. If the build is broken and the tests are not flaky, we can vote for disabling those tests. I think that vote will pass. I hope we understand what this project is now. One person can make a big mess if the guards are not set up correctly. Five more companies can come in and start working on this, out ruling everyone we have now. The base has to be solid for many reasons. Letting this change in as a rule opens us to too much abuse, intentional or unintentional. Toni On Thu, Jan 25, 2024 at 2:59 PM Francisco Javier Tirado Sarti < ftira...@redhat.com> wrote: > +1 > Thanks Tibor > > On Thu, Jan 25, 2024 at 9:37 AM Tibor Zimányi <tzima...@apache.org> wrote: > > > Hi, > > > > as I wrote in one of the previous e-mails in this thread: > > "I think the notification of the involved commiter could be done as part > of > > creating of the issue to fix the tests. The person involved can be > assigned > > as an asignee. " > > > > To clarify, by involved commiter I meant either author of the test or the > > person who is maintaining that part of the codebase. > > > > Hopefully this clarifies this. > > > > Tibor > > > > Dňa št 25. 1. 2024, 9:22 Francisco Javier Tirado Sarti < > > ftira...@redhat.com> > > napísal(a): > > > > > I agree with Josef, I realize the SpringBoot listener example test was > > > disabled while fixing an issue related to it. If not, the test would > have > > > remained disabled. > > > > > > On Thu, Jan 25, 2024 at 5:51 AM Jozef Marko > <jozef.ma...@ibm.com.invalid > > > > > > wrote: > > > > > > > Thank you for this discussion. I think disabling flaky tests in given > > way > > > > is reasonable. > > > > > > > > I have one related question, once similar flaky test is disabled, > and a > > > > ticket is reported. Who will be assigned to this ticket? Will it be > the > > > > test author? > > > > > > > > My point is, that usually only test author and few other engineers > > > working > > > > on the same component understand completely the given test, but > > probably > > > > they do not check the codebase daily if the test is still enabled. > > > > > > > > So, will be these people notified? Otherwise, most of these tickets > > will > > > > be probably closed after the proposed period without any test > > update/fix. > > > > > > > > Jozef Marko > > > > Software Developer > > > > jozef.ma...@ibm.com > > > > > > > > -----Original Message----- > > > > From: Toni Rikkola <trikk...@redhat.com> > > > > Sent: Wednesday, January 24, 2024 3:58 PM > > > > To: dev@kie.apache.org > > > > Subject: [EXTERNAL] Re: [PROPOSAL] Workflow for ignoring long failing > > > tests > > > > > > > > The reason I mentioned this was that we had this same decision made 8 > > > > years ago and shared it to the entire team. > > > > It stated that anyone can disable any flaky test and make a ticket > for > > > it. > > > > It was even more strict, you just had to have one flaky failure. > > > > > > > > So to me it looks like we are just stating the same thing again > hoping > > > for > > > > a different outcome. > > > > > > > > But yes, to make it clear for everyone we should write it down and we > > can > > > > see how it goes and act on it later if needed. > > > > > > > > Toni > > > > > > > > On Wed, Jan 24, 2024 at 2:09 PM Tibor Zimányi <tzima...@apache.org> > > > wrote: > > > > > > > > > I think we don't need to enforce it or gatekeep this any way. The > > > > > thing is, if we have the agreement that we want to have this > official > > > > > workflow, it should fix the problem you are mentioning Toni, that > > > > > people are afraid to do so. If it will be officially stated, that > > > > > failing tests can be disabled in the code, people should just do > it. > > > > > They can always point to this potential agreement. The best thing > > > > > would be to even document it on Confluence. I will do that if we > have > > > an > > > > agreement on this. > > > > > > > > > > Tibor > > > > > > > > > > Dňa st 24. 1. 2024, 12:40 Toni Rikkola <trikk...@redhat.com> > > > napísal(a): > > > > > > > > > > > Not informing the author was my mistake on that PR. I should have > > > > > > asked > > > > > the > > > > > > original author to review it. > > > > > > > > > > > > But the problem is not that we lack this procedure. The KIE team > > > > > > agreed > > > > > on > > > > > > this in 2015. It is that maybe we lack the strength to do it, > > > > > > especially when it is someone else's area of code. This needs > some > > > > > > sort of > > > > > gatekeeper > > > > > > or supervisor and time frames when the build quality and if these > > > > > problems > > > > > > were acted on are validated and measured. > > > > > > > > > > > > Meanwhile I offer myself to work on these and disable them, if > > > > > > anyone > > > > > wants > > > > > > to point me to some flaky test I can act on them. I could even > make > > > > > > some system for measurement. We can then, or now, decide what we > > > > > > then do with that data. > > > > > > > > > > > > Toni > > > > > > > > > > > > On Tue, Jan 23, 2024 at 8:04 PM Tibor Zimányi < > tzima...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > I think the notification of the involved commiter could be done > > as > > > > > > > part > > > > > > of > > > > > > > creating of the issue to fix the tests. The person involved can > > be > > > > > > assigned > > > > > > > as an asignee. > > > > > > > > > > > > > > T. > > > > > > > > > > > > > > Dňa ut 23. 1. 2024, 18:50 Francisco Javier Tirado Sarti < > > > > > > > ftira...@redhat.com> > > > > > > > napísal(a): > > > > > > > > > > > > > > > Ok, but I think we first need to establish a way (my > apologies > > > > > > > > is > > > > > that > > > > > > > way > > > > > > > > is already there) to notify committerthat a test that they > > were > > > > > > involved > > > > > > > > with is failing > > > > > > > > That way, we will avoid disable tests like this one > > > > > > > > INVALID URI REMOVED > > > > > > > > > > apache_incubator-2Dkie-2Dkogito-2Dexamples_issues_1831&d=DwIFaQ& > > > > > > > > > > c=jf_iaSHvJObTbx-siA1ZOg&r=Tvtr6trQuIEn20iSHtFwcpynBt8ViCybz4kYp > > > > > > > > > > jmDDUQ&m=C6gAvciwgXz7uJX__01AVZ3YHxSotIhuktwbnBnNuTvnLRU-NHMRqOd > > > > > > > > NENKhsqKe&s=8p3WYlpcXfcLl1n4BnrSpBWNIdPIUhslTPsb41eE4IM&e= , > > > > > > > which > > > > > > > > are quite important to ensure Springboot messaging is > working. > > > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 6:38 PM ricardo zanini fernandes < > > > > > > > > ricardozan...@gmail.com> wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 1:03 PM Martin Cimbálek > > > > > > > > > <cim...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > Martin Cimbalek > > > > > > > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 3:36 PM Tibor Zimányi < > > > > > tzima...@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > I want to propose a workflow for situations, when some > > > > > > > > > > > tests > > > > > are > > > > > > > > > failing > > > > > > > > > > > for a longer time. In such cases, my proposed workflow > > is: > > > > > > > > > > > - If a test or a set of tests is failing for two days > > > > > (nightlies > > > > > > or > > > > > > > > PR > > > > > > > > > > > checks), ignore those tests in the codebase, so they > are > > > > > > > > > > > not > > > > > > > > executed. > > > > > > > > > > > - File an issue in kie-issues repository, reporting the > > > > > > > > > > > test > > > > > > > > failures. > > > > > > > > > > > - If the issue is not resolved for half a year, delete > > > > > > > > > > > those > > > > > > tests > > > > > > > > from > > > > > > > > > > the > > > > > > > > > > > codebase. > > > > > > > > > > > > > > > > > > > > > > What do you think please? This should make sure all > > > > > > > > > > > failures > > > > > that > > > > > > > > don't > > > > > > > > > > get > > > > > > > > > > > fixed immediately after they occur get logged in the > > issue > > > > > > tracker, > > > > > > > > so > > > > > > > > > > they > > > > > > > > > > > can be appropriately handled and don't block unrelated > PR > > > > > checks > > > > > > > and > > > > > > > > > > > builds. It will also make sure that tests that are not > > > > > > > > > > > fixed > > > > > for > > > > > > a > > > > > > > > long > > > > > > > > > > > time (therefore they could be perceived as not > important) > > > > > > > > > > > are > > > > > > not a > > > > > > > > > > > maintenance burden for the future. > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > Tibor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > >
