Re: Fixing flaky tests in Apache Hadoop

2020-10-23 Thread Ahmed Hussein
> > 2. Dedicate a community "Bug Bash Day" / "Fix it Day". We had a bug bash > day two years ago, and maybe it's time to repeat it again: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75965105 > this > is going to be tricky as we are in a pandemic and most of the community

Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Ahmed Hussein
Thank you Akira and We-Chiu. IMHO, the citation is more than just flaky tests. It has more depth: - Every developer stays committed to keep the code healthy. - Those flaky tests are actually "*bugs*" that need to be fixed. It is evident that there is a major problem in handling the resources as

Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Wei-Chiu Chuang
I also wondered if the hardware was too stressed since all Hadoop related projects all use the same set of Jenkins servers. However, HBase just recently moved to their own dedicated machines, so I'm actually surprised to see a lot of resource related failures even now. On Thu, Oct 22, 2020 at

Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Wei-Chiu Chuang
Thanks for raising the issue, Akira and Ahmed, Fixing flaky tests is a thankless job so I want to take this opportunity to recognize the time and effort. We will always have flaky tests due to bad tests or simply infra issues. Fixing flaky tests will take time but if they are not addressed it

Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Akira Ajisaka
Hi Hadoop developers, Now there are a lot of failing unit tests and there is an issue to tackle this bad situation. https://issues.apache.org/jira/browse/HDFS-15646 Although this issue is in HDFS project, this issue is related to all the Hadoop developers. Please check the above URL, read the