Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Ahmed Hussein
Thank you Akira and We-Chiu. IMHO, the citation is more than just flaky tests. It has more depth: - Every developer stays committed to keep the code healthy. - Those flaky tests are actually "*bugs*" that need to be fixed. It is evident that there is a major problem in handling the resources as

[jira] [Created] (HADOOP-17325) WASB: Test failure in trunk

2020-10-22 Thread Sneha Vijayarajan (Jira)
Sneha Vijayarajan created HADOOP-17325: -- Summary: WASB: Test failure in trunk Key: HADOOP-17325 URL: https://issues.apache.org/jira/browse/HADOOP-17325 Project: Hadoop Common Issue

Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Wei-Chiu Chuang
I also wondered if the hardware was too stressed since all Hadoop related projects all use the same set of Jenkins servers. However, HBase just recently moved to their own dedicated machines, so I'm actually surprised to see a lot of resource related failures even now. On Thu, Oct 22, 2020 at

[jira] [Created] (HADOOP-17324) ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException

2020-10-22 Thread Chao Sun (Jira)
Chao Sun created HADOOP-17324: - Summary: ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException Key: HADOOP-17324 URL: https://issues.apache.org/jira/browse/HADOOP-17324

Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64

2020-10-22 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/35/ [Oct 21, 2020 5:59:35 AM] (Kihwal Lee) HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. [Oct 21, 2020 12:01:13 PM] (noreply) HADOOP-17305. Fix ITestCustomSigner to work

Re: Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Wei-Chiu Chuang
Thanks for raising the issue, Akira and Ahmed, Fixing flaky tests is a thankless job so I want to take this opportunity to recognize the time and effort. We will always have flaky tests due to bad tests or simply infra issues. Fixing flaky tests will take time but if they are not addressed it

[jira] [Resolved] (HADOOP-17319) Update the checkstyle config to ban some guava functions

2020-10-22 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17319. Fix Version/s: 3.4.0 Resolution: Fixed Merged the PR into trunk. > Update the

Fixing flaky tests in Apache Hadoop

2020-10-22 Thread Akira Ajisaka
Hi Hadoop developers, Now there are a lot of failing unit tests and there is an issue to tackle this bad situation. https://issues.apache.org/jira/browse/HDFS-15646 Although this issue is in HDFS project, this issue is related to all the Hadoop developers. Please check the above URL, read the

[jira] [Created] (HADOOP-17323) s3a getFileStatus("/") to skip IO

2020-10-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17323: --- Summary: s3a getFileStatus("/") to skip IO Key: HADOOP-17323 URL: https://issues.apache.org/jira/browse/HADOOP-17323 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-17322) TestZKFailoverController#testNoZK is flaky

2020-10-22 Thread Akira Ajisaka (Jira)
Akira Ajisaka created HADOOP-17322: -- Summary: TestZKFailoverController#testNoZK is flaky Key: HADOOP-17322 URL: https://issues.apache.org/jira/browse/HADOOP-17322 Project: Hadoop Common

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

2020-10-22 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/302/ [Oct 21, 2020 5:59:35 AM] (Kihwal Lee) HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. [Oct 21, 2020 12:01:13 PM] (noreply) HADOOP-17305. Fix ITestCustomSigner to work

[jira] [Resolved] (HADOOP-16780) Track unstable tests according to aarch CI due to OOM

2020-10-22 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein resolved HADOOP-16780. Resolution: Resolved > Track unstable tests according to aarch CI due to OOM >

[jira] [Reopened] (HADOOP-16780) Track unstable tests according to aarch CI due to OOM

2020-10-22 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reopened HADOOP-16780: > Track unstable tests according to aarch CI due to OOM >

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

2020-10-22 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/93/ No changes -1 overall The following subsystems voted -1: asflicense hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored:

Re: Quality Outreach - Survey

2020-10-22 Thread Rory O'Donnell
Many thanks for your feedback Akira! On 22/10/2020 09:45, Akira Ajisaka wrote: +cc: common-dev@hadoop.apache.org 1) Does your project support JDK 15 (b) No, but we plan to support in future. 2) Have you tested JDK 16 EA Builds ? (b) No, but we plan to test JDK 16 EA builds in the future 3)

Re: Quality Outreach - Survey

2020-10-22 Thread Akira Ajisaka
+cc: common-dev@hadoop.apache.org 1) Does your project support JDK 15 (b) No, but we plan to support in future. 2) Have you tested JDK 16 EA Builds ? (b) No, but we plan to test JDK 16 EA builds in the future 3) Do you find the emails/communication sent as part of Quality Outreach useful ? (a)

Multi-node cluster tutorial

2020-10-22 Thread Johnny Nguyen
Hello, My name is Johnny and I am happy to setup a hadoop cluster with 3 computers: - hadoop-master: ubuntu 20.04 on Windows (WSL)* - hadoop-slave1: ubuntu 18.04 on Linux - hadoop-slave2: ubuntu 20.04 on Windows (WSL)* I know how to easely do that and I would like to contribute on the page