The reason I mentioned this was that we had this same decision made 8 years ago and shared it to the entire team. It stated that anyone can disable any flaky test and make a ticket for it. It was even more strict, you just had to have one flaky failure.
So to me it looks like we are just stating the same thing again hoping for a different outcome. But yes, to make it clear for everyone we should write it down and we can see how it goes and act on it later if needed. Toni On Wed, Jan 24, 2024 at 2:09 PM Tibor Zimányi <tzima...@apache.org> wrote: > I think we don't need to enforce it or gatekeep this any way. The thing is, > if we have the agreement that we want to have this official workflow, it > should fix the problem you are mentioning Toni, that people are afraid to > do so. If it will be officially stated, that failing tests can be disabled > in the code, people should just do it. They can always point to this > potential agreement. The best thing would be to even document it on > Confluence. I will do that if we have an agreement on this. > > Tibor > > Dňa st 24. 1. 2024, 12:40 Toni Rikkola <trikk...@redhat.com> napísal(a): > > > Not informing the author was my mistake on that PR. I should have asked > the > > original author to review it. > > > > But the problem is not that we lack this procedure. The KIE team agreed > on > > this in 2015. It is that maybe we lack the strength to do it, especially > > when it is someone else's area of code. This needs some sort of > gatekeeper > > or supervisor and time frames when the build quality and if these > problems > > were acted on are validated and measured. > > > > Meanwhile I offer myself to work on these and disable them, if anyone > wants > > to point me to some flaky test I can act on them. I could even make some > > system for measurement. We can then, or now, decide what we then do with > > that data. > > > > Toni > > > > On Tue, Jan 23, 2024 at 8:04 PM Tibor Zimányi <tzima...@apache.org> > wrote: > > > > > I think the notification of the involved commiter could be done as part > > of > > > creating of the issue to fix the tests. The person involved can be > > assigned > > > as an asignee. > > > > > > T. > > > > > > Dňa ut 23. 1. 2024, 18:50 Francisco Javier Tirado Sarti < > > > ftira...@redhat.com> > > > napísal(a): > > > > > > > Ok, but I think we first need to establish a way (my apologies is > that > > > way > > > > is already there) to notify committerthat a test that they were > > involved > > > > with is failing > > > > That way, we will avoid disable tests like this one > > > > https://github.com/apache/incubator-kie-kogito-examples/issues/1831, > > > which > > > > are quite important to ensure Springboot messaging is working. > > > > > > > > On Tue, Jan 23, 2024 at 6:38 PM ricardo zanini fernandes < > > > > ricardozan...@gmail.com> wrote: > > > > > > > > > +1 > > > > > > > > > > On Tue, Jan 23, 2024 at 1:03 PM Martin Cimbálek <cim...@gmail.com> > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > Regards, > > > > > > Martin Cimbalek > > > > > > > > > > > > On Tue, Jan 23, 2024 at 3:36 PM Tibor Zimányi < > tzima...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > I want to propose a workflow for situations, when some tests > are > > > > > failing > > > > > > > for a longer time. In such cases, my proposed workflow is: > > > > > > > - If a test or a set of tests is failing for two days > (nightlies > > or > > > > PR > > > > > > > checks), ignore those tests in the codebase, so they are not > > > > executed. > > > > > > > - File an issue in kie-issues repository, reporting the test > > > > failures. > > > > > > > - If the issue is not resolved for half a year, delete those > > tests > > > > from > > > > > > the > > > > > > > codebase. > > > > > > > > > > > > > > What do you think please? This should make sure all failures > that > > > > don't > > > > > > get > > > > > > > fixed immediately after they occur get logged in the issue > > tracker, > > > > so > > > > > > they > > > > > > > can be appropriately handled and don't block unrelated PR > checks > > > and > > > > > > > builds. It will also make sure that tests that are not fixed > for > > a > > > > long > > > > > > > time (therefore they could be perceived as not important) are > > not a > > > > > > > maintenance burden for the future. > > > > > > > > > > > > > > Best regards, > > > > > > > Tibor > > > > > > > > > > > > > > > > > > > > > > > > > > > >
