Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Surendra Singh Lilhore
Thanks Wei-Chiu, I feel now IBR is more stable in branch 3.x. If BR is just added to prevent bugs in IBR, I feel we should fix such bug in IBR. Adding one new functionality to prevent bug in other is not good. I also thing, DN should send BR in failure and process start scenario only. -Surendra

Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Ayush Saxena
Hi Wei-Chiu, Thanx for the response. Yes, We are talking about the FBR only. Increasing the frequency limits the problem, but doesn’t seems to be solving it. With increasing cluster size, the frequency needs to be increased, and we cannot increase it indefinitely, as in some case FBR is needed. O

Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Wei-Chiu Chuang
Hey Ayush, Thanks a lot for your proposal. Do you mean the Full Block Report that is sent out every 6 hours per DataNode? Someone told me they reduced the frequency of FBR to 24 hours and it seems okay. One of the purposes of FBR was to prevent bugs in incremental block report implementation. In

[jira] [Resolved] (HADOOP-16845) ITestAbfsClient.testContinuationTokenHavingEqualSign failing

2020-02-06 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt resolved HADOOP-16845. -- Fix Version/s: 3.3.0 Resolution: Fixed commit 55f2421580678a6793c8cb6ad10fee3f

[jira] [Resolved] (HADOOP-16825) ITestAzureBlobFileSystemCheckAccess failing

2020-02-06 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt resolved HADOOP-16825. -- Fix Version/s: 3.3.0 Resolution: Fixed commit 5944d28130925fe1452f545e96b5e44f

[jira] [Created] (HADOOP-16847) Test can fail if HashSet iterates in a different order

2020-02-06 Thread testfixer0 (Jira)
testfixer0 created HADOOP-16847: --- Summary: Test can fail if HashSet iterates in a different order Key: HADOOP-16847 URL: https://issues.apache.org/jira/browse/HADOOP-16847 Project: Hadoop Common

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2020-02-06 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1403/ [Feb 5, 2020 1:44:05 AM] (jhung) YARN-10116. Expose diagnostics in RMAppManager summary -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsyste

[jira] [Created] (HADOOP-16846) add experimental optimization of s3a directory marker handling

2020-02-06 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-16846: --- Summary: add experimental optimization of s3a directory marker handling Key: HADOOP-16846 URL: https://issues.apache.org/jira/browse/HADOOP-16846 Project: Hadoo

[jira] [Resolved] (HADOOP-16832) S3Guard testing doc: Add required parameters for S3Guard testing in IDE

2020-02-06 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota resolved HADOOP-16832. - Resolution: Fixed > S3Guard testing doc: Add required parameters for S3Guard testing in IDE > -

Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86

2020-02-06 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/589/ [Feb 5, 2020 7:16:09 PM] (jhung) YARN-10116. Expose diagnostics in RMAppManager summary [Feb 5, 2020 7:33:30 PM] (kihwal) HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. -