> I've been using this type of template for reporting a flaky test: https://gist.github.com/lhotari/a5c67359b362b4f3d8729330d65a2298 .
Can you add an issue template to Pulsar? - Sijie On Fri, Jan 29, 2021 at 10:27 AM Lari Hotari <lari.hot...@sagire.fi> wrote: > Dear Pulsar community members, > > In order to improve our CI, we will have to fix the flaky tests. In some > cases it might be necessary to replace an existing test with a redesigned > test. > > The draft PIP "Changes to flaky test handling" document > < > https://docs.google.com/document/d/10lmn4pW1IsT_8D1ZE0vMjASX0HhjdGdjB794iyScwns/edit?usp=sharing > > > lists > the top 10 flaky tests. A lot of them have already been address by pull > requests in the past week or so. > > This is the list of recent PRs that fix flaky tests from the top 10 flaky > tests list: > https://github.com/apache/pulsar/pull/9286 > https://github.com/apache/pulsar/pull/9243 > https://github.com/apache/pulsar/pull/9258 > https://github.com/apache/pulsar/pull/9356 > > These are the GH issues for the remaining ones in the top 10 flaky tests > list: > https://github.com/apache/pulsar/issues/6368 > https://github.com/apache/pulsar/issues/9369 > https://github.com/apache/pulsar/issues/9368 > > If you would like to help to fix flaky tests you can pick one of the open > issues above. Just add a comment on the issue when you start working on it > so that we can coordinate activities. > > It is also helpful to report a flaky test when you encounter one. I've been > using this type of template for reporting a flaky test: > https://gist.github.com/lhotari/a5c67359b362b4f3d8729330d65a2298 . The > issues #9368 and #9369 have been reported using this template. > Search for the test name before reporting so that we don't end up with > duplicates. > > The issues #6368, #9369 and #9368 are the 3 next important issues to fix. > I'm planning to create a more extensive list of the flaky failures so that > we can target the most flaky ones when we continue fixing the flaky tests. > I have some scripts in development to assist in mining the Pulsar Github > Action workflow run logs. > > This is a search to find flaky issues in Pulsar GH issues: > > https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen > > Looking forward to the contributions for fixing flaky tests, > > BR, > > Lari >
