Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/328/ No changes ERROR: File 'out/email-report.txt' does not exist - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/ [Jun 11, 2021 3:15:52 AM] (noreply) HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087) [Jun 11, 2021 7:34:24 AM] (Mingliang Liu) Revert "HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)" [Jun 11, 2021 8:30:55 AM] (noreply) HDFS-16057. Make sure the order for location in ENTERING_MAINTENANCE state (#3084) [Jun 11, 2021 4:00:42 PM] (noreply) HDFS-15671. testBalancerRPCDelayQpsDefault fails intermittently (#3097) -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl hadoop.yarn.csi.client.TestCsiClient cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-compile-cc-root.txt [96K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-compile-javac-root.txt [376K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/blanks-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-pathlen.txt [16K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-shellcheck.txt [28K] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/results-javadoc-javadoc-root.txt [408K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [516K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/536/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt [20K] Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.3.1 RC3
+1 (Binding) 1. Built from Tag. 2. Successful Native Build on Ubuntu 20.04 3. Verified Checksums 4. Deployed the docker cluster with 3 nodes 5. Ran sample MR Jobs -Vinay On Sat, Jun 12, 2021 at 6:40 PM Ayush Saxena wrote: > +1, > Built from Source. > Successful Native Build on Ubuntu 20.04 > Verified Checksums > Ran basic hdfs shell commands. > Ran simple MR jobs. > Browsed NN,DN,RM and NM UI. > > Thanx Wei-Chiu for driving the release. > > -Ayush > > > > On 12-Jun-2021, at 1:45 AM, epa...@apache.org wrote: > > > > +1 (binding) > > Eric > > > > > > On Tuesday, June 1, 2021, 5:29:49 AM CDT, Wei-Chiu Chuang < > weic...@apache.org> wrote: > > > > Hi community, > > > > This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All > blocker > > issues have been resolved [1] again. > > > > There are 2 additional issues resolved for RC3: > > * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after > > HADOOP-16878 > > * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the > source > > and destination are the same > > > > There are 4 issues resolved for RC2: > > * HADOOP-17666. Update LICENSE for 3.3.1 > > * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053) > > * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) > > * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) > > > > The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two extra > > fixes compared to hadoop-thirdparty 1.1.0: > > * HADOOP-17707. Remove jaeger document from site index. > > * HADOOP-17730. Add back error_prone > > > > *RC tag is release-3.3.1-RC3 > > https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3 > > > > *The RC3 artifacts are at*: > > https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/ > > ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/ > > > > *The maven artifacts are hosted here:* > > https://repository.apache.org/content/repositories/orgapachehadoop-1320/ > > > > *My public key is available here:* > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > Things I've verified: > > * all blocker issues targeting 3.3.1 have been resolved. > > * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible. > > * LICENSE and NOTICE files checked > > * RELEASENOTES and CHANGELOG > > * rat check passed. > > * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests. > > * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests. > > * Extra: built 50 other open source projects on top of Hadoop 3.3.1 RC2. > > Had to patch some of them due to commons-lang migration (Hadoop 3.2.0) > and > > dependency divergence. Issues are being identified but so far nothing > > blocker for Hadoop itself. > > > > Please try the release and vote. The vote will run for 5 days. > > > > My +1 to start, > > > > [1] https://issues.apache.org/jira/issues/?filter=12350491 > > [2] > > > https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3 > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
[jira] [Resolved] (HADOOP-17547) Magic committer to downgrade abort in cleanup if list uploads fails with access denied
[
https://issues.apache.org/jira/browse/HADOOP-17547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-17547.
-
Resolution: Fixed
fixed by downgrading to log at debug
> Magic committer to downgrade abort in cleanup if list uploads fails with
> access denied
> --
>
> Key: HADOOP-17547
> URL: https://issues.apache.org/jira/browse/HADOOP-17547
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Bogdan Stolojan
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If the caller doesn't have "s3:ListBucketMultipartUploads" permissions on a
> bucket, then magic committer cleanup fails.
> {code}
> at
> org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:247)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:112)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:315)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:407)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:311)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:286)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.listMultipartUploads(S3AFileSystem.java:4549)
> at
> org.apache.hadoop.fs.s3a.commit.CommitOperations.listPendingUploadsUnderPath(CommitOperations.java:361)
> at
> org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.abortPendingUploadsInCleanup(AbstractS3ACommitter.java:671)
> at
> org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.cleanup(AbstractS3ACommitter.java:770)
> {code}
> it should just swallow this, given it's best effort
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Change project style guidelines to allow line length 100
+1 if you look closely the hadoop-azure module went to 100 lines a while back and all is good On Wed, 19 May 2021 at 22:13, Sean Busbey wrote: > Hello! > > What do folks think about changing our line length guidelines to allow for > 100 character width? > > Currently, we tell folks to follow the sun style guide with some exception > unrelated to line length. That guide says width of 80 is the standard and > our current check style rules act as enforcement. > > Looking at the current trunk codebase our nightly build shows a total of > ~15k line length violations; it’s about 18% of identified checkstyle issues. > > The vast majority of those line length violations are <= 100 characters > long. 100 characters happens to be the length for the Google Java Style > Guide, another commonly adopted style guide for java projects, so I suspect > these longer lines leaking past the checkstyle precommit warning might be a > reflection of committers working across multiple java codebases. > > I don’t feel strongly about lines being longer, but I would like to move > towards more consistent style enforcement as a project. Updating our > project guidance to allow for 100 character lines would reduce the > likelihood that folks bringing in new contributions need a precommit test > cycle to get the formatting correct. > > Does anyone feel strongly about keeping the line length limit at 80 > characters? > > Does anyone feel strongly about contributions coming in that clear up line > length violations? > > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > >
Re: [VOTE] Release Apache Hadoop 3.3.1 RC3
+1, Built from Source. Successful Native Build on Ubuntu 20.04 Verified Checksums Ran basic hdfs shell commands. Ran simple MR jobs. Browsed NN,DN,RM and NM UI. Thanx Wei-Chiu for driving the release. -Ayush > On 12-Jun-2021, at 1:45 AM, epa...@apache.org wrote: > > +1 (binding) > Eric > > > On Tuesday, June 1, 2021, 5:29:49 AM CDT, Wei-Chiu Chuang > wrote: > > Hi community, > > This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker > issues have been resolved [1] again. > > There are 2 additional issues resolved for RC3: > * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after > HADOOP-16878 > * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source > and destination are the same > > There are 4 issues resolved for RC2: > * HADOOP-17666. Update LICENSE for 3.3.1 > * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053) > * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) > * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) > > The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two extra > fixes compared to hadoop-thirdparty 1.1.0: > * HADOOP-17707. Remove jaeger document from site index. > * HADOOP-17730. Add back error_prone > > *RC tag is release-3.3.1-RC3 > https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3 > > *The RC3 artifacts are at*: > https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/ > ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/ > > *The maven artifacts are hosted here:* > https://repository.apache.org/content/repositories/orgapachehadoop-1320/ > > *My public key is available here:* > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > Things I've verified: > * all blocker issues targeting 3.3.1 have been resolved. > * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible. > * LICENSE and NOTICE files checked > * RELEASENOTES and CHANGELOG > * rat check passed. > * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests. > * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests. > * Extra: built 50 other open source projects on top of Hadoop 3.3.1 RC2. > Had to patch some of them due to commons-lang migration (Hadoop 3.2.0) and > dependency divergence. Issues are being identified but so far nothing > blocker for Hadoop itself. > > Please try the release and vote. The vote will run for 5 days. > > My +1 to start, > > [1] https://issues.apache.org/jira/issues/?filter=12350491 > [2] > https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3 - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [E] Re: [DISCUSS] Change project style guidelines to allow line length 100
Thanks for this initiative, Sean. +1 for increasing the length to 100 characters. I can't see a VOTE thread regarding this subject. Am I missing something? Best, Szilard On Mon, May 24, 2021 at 11:49 PM Jonathan Eagles wrote: > In apache tez, formal line length is 120 characters. So, I recommend 120+ > > On Mon, May 24, 2021 at 4:46 PM Kihwal Lee .invalid> > wrote: > > > +1 for the 100 char limit. > > But I would have liked 132 columns more. :) > > > > Kihwal > > > > On Mon, May 24, 2021 at 1:46 PM Sean Busbey > > wrote: > > > > > Hi folks! > > > > > > The consensus seems pretty strongly in favor of increasing the line > > length > > > limit. Do folks still want to see a formal VOTE thread? > > > > > > > > > > On May 19, 2021, at 4:22 PM, Sean Busbey > > > wrote: > > > > > > > > Hello! > > > > > > > > What do folks think about changing our line length guidelines to > allow > > > for 100 character width? > > > > > > > > Currently, we tell folks to follow the sun style guide with some > > > exception unrelated to line length. That guide says width of 80 is the > > > standard and our current check style rules act as enforcement. > > > > > > > > Looking at the current trunk codebase our nightly build shows a total > > of > > > ~15k line length violations; it’s about 18% of identified checkstyle > > issues. > > > > > > > > The vast majority of those line length violations are <= 100 > characters > > > long. 100 characters happens to be the length for the Google Java Style > > > Guide, another commonly adopted style guide for java projects, so I > > suspect > > > these longer lines leaking past the checkstyle precommit warning might > > be a > > > reflection of committers working across multiple java codebases. > > > > > > > > I don’t feel strongly about lines being longer, but I would like to > > move > > > towards more consistent style enforcement as a project. Updating our > > > project guidance to allow for 100 character lines would reduce the > > > likelihood that folks bringing in new contributions need a precommit > test > > > cycle to get the formatting correct. > > > > > > > > Does anyone feel strongly about keeping the line length limit at 80 > > > characters? > > > > > > > > Does anyone feel strongly about contributions coming in that clear up > > > line length violations? > > > > > > > > > > - > > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > > > > > >
