[jira] [Created] (HADOOP-17123) remove guava Preconditions from Hadoop-common module
Ahmed Hussein created HADOOP-17123: -- Summary: remove guava Preconditions from Hadoop-common module Key: HADOOP-17123 URL: https://issues.apache.org/jira/browse/HADOOP-17123 Project: Hadoop Common Issue Type: Sub-task Components: common Reporter: Ahmed Hussein Assignee: Ahmed Hussein Replace guava Preconditions by internal implementations that rely on java8+ APIs in the hadoop -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/744/ [Jul 10, 2020 4:38:08 AM] (iwasakims) HADOOP-12940. Fix warnings from Spotbugs in hadoop-common. [Jul 10, 2020 5:37:05 AM] (iwasakims) HADOOP-14354. SysInfoWindows is not thread safe. Contributed by Inigo [Jul 10, 2020 6:16:51 AM] (iwasakims) HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by -1 overall The following subsystems voted -1: asflicense findbugs hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml findbugs : module:hadoop-yarn-project/hadoop-yarn Useless object stored in variable removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:[line 664] org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() makes inefficient use of keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:[line 741] org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() makes inefficient use of keySet iterator instead of entrySet iterator At ContainerLocalizer.java:keySet iterator instead of entrySet iterator At ContainerLocalizer.java:[line 359] org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics is a mutable collection which should be package protected At ContainerMetrics.java:which should be package protected At ContainerMetrics.java:[line 134] Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] org.apache.hadoop.yarn.state.StateMachineFactory.generateStateGraph(String) makes inefficient use of keySet iterator instead of entrySet iterator At StateMachineFactory.java:keySet iterator instead of entrySet iterator At StateMachineFactory.java:[line 505] findbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common org.apache.hadoop.yarn.state.StateMachineFactory.generateStateGraph(String) makes inefficient use of keySet iterator instead of entrySet iterator At StateMachineFactory.java:keySet iterator instead of entrySet iterator At StateMachineFactory.java:[line 505] findbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Useless object stored in variable removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:[line 664] org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() makes inefficient use of keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:[line 741] org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() makes inefficient use of keySet iterator instead of entrySet iterator At ContainerLocalizer.java:keySet iterator instead of entrySet iterator At ContainerLocalizer.java:[line 359] org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics is a mutable collection which should be package protected At ContainerMetrics.java:which should be package protected At ContainerMetrics.java:[line 134] Boxed value is u
Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0
+1(non-binding) * Built from source on Ubuntu-18.04 (ARM & x86) * Successful native build. * Ran some basic HDFS shell commands. * Browsed through the WebUI (NN, DN, RM & NM) * Executed WordCount, TeraGen, TeraSort & TeraValidate * Verified checksums and signatures. Thanx everyone for the efforts towards the release. Good Luck!!! -Ayush On Sat, 11 Jul 2020 at 00:05, Iñigo Goiri wrote: > +1 (Binding) > > Deployed a cluster on Azure VMs with: > * 3 VMs with HDFS Namenodes and Routers > * 2 VMs with YARN Resource Managers > * 5 VMs with HDFS Datanodes and Node Managers > > Tests: > * Executed Tergagen+Terasort+Teravalidate. > * Executed wordcount. > * Browsed through the Web UI. > > > > On Fri, Jul 10, 2020 at 1:06 AM Vinayakumar B > wrote: > > > +1 (Binding) > > > > -Verified all checksums and Signatures. > > -Verified site, Release notes and Change logs > > + May be changelog and release notes could be grouped based on the > > project at second level for better look (this needs to be supported from > > yetus) > > -Tested in x86 local 3-node docker cluster. > > + Built from source with OpenJdk 8 and Ubuntu 18.04 > > + Deployed 3 node docker cluster > > + Ran various Jobs (wordcount, Terasort, Pi, etc) > > > > No Issues reported. > > > > -Vinay > > > > On Fri, Jul 10, 2020 at 1:19 PM Sheng Liu > wrote: > > > > > +1 (non-binding) > > > > > > - checkout the "3.3.0-aarch64-RC0" binaries packages > > > > > > - started a clusters with 3 nodes VMs of Ubuntu 18.04 ARM/aarch64, > > > openjdk-11-jdk > > > > > > - checked some web UIs (NN, DN, RM, NM) > > > > > > - Executed a wordcount, TeraGen, TeraSort and TeraValidate > > > > > > - Executed a TestDFSIO job > > > > > > - Executed a Pi job > > > > > > BR, > > > Liusheng > > > > > > Zhenyu Zheng 于2020年7月10日周五 下午3:45写道： > > > > > > > +1 (non-binding) > > > > > > > > - Verified all hashes and checksums > > > > - Tested on ARM platform for the following actions: > > > > + Built from source on Ubuntu 18.04, OpenJDK 8 > > > > + Deployed a pseudo cluster > > > > + Ran some example jobs(grep, wordcount, pi) > > > > + Ran teragen/terasort/teravalidate > > > > + Ran TestDFSIO job > > > > > > > > BR, > > > > > > > > Zhenyu > > > > > > > > On Fri, Jul 10, 2020 at 2:40 PM Akira Ajisaka > > > wrote: > > > > > > > > > +1 (binding) > > > > > > > > > > - Verified checksums and signatures. > > > > > - Built from the source with CentOS 7 and OpenJDK 8. > > > > > - Successfully upgraded HDFS to 3.3.0-RC0 in our development > cluster > > > > (with > > > > > RBF, security, and OpenJDK 11) for end-users. No issues reported. > > > > > - The document looks good. > > > > > - Deployed pseudo cluster and ran some MapReduce jobs. > > > > > > > > > > Thanks, > > > > > Akira > > > > > > > > > > > > > > > On Tue, Jul 7, 2020 at 7:27 AM Brahma Reddy Battula < > > bra...@apache.org > > > > > > > > > wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > This is the first release candidate for the first release of > Apache > > > > > > Hadoop 3.3.0 > > > > > > line. > > > > > > > > > > > > It contains *1644[1]* fixed jira issues since 3.2.1 which > include a > > > lot > > > > > of > > > > > > features and improvements(read the full set of release notes). > > > > > > > > > > > > Below feature additions are the highlights of the release. > > > > > > > > > > > > - ARM Support > > > > > > - Enhancements and new features on S3a,S3Guard,ABFS > > > > > > - Java 11 Runtime support and TLS 1.3. > > > > > > - Support Tencent Cloud COS File System. > > > > > > - Added security to HDFS Router. > > > > > > - Support non-volatile storage class memory(SCM) in HDFS cache > > > > directives > > > > > > - Support Interactive Docker Shell for running Containers. > > > > > > - Scheduling of opportunistic containers > > > > > > - A pluggable device plugin framework to ease vendor plugin > > > development > > > > > > > > > > > > *The RC0 artifacts are at*: > > > > > > http://home.apache.org/~brahma/Hadoop-3.3.0-RC0/ > > > > > > > > > > > > *First release to include ARM binary, Have a check.* > > > > > > *RC tag is *release-3.3.0-RC0. > > > > > > > > > > > > > > > > > > *The maven artifacts are hosted here:* > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1271/ > > > > > > > > > > > > *My public key is available here:* > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > > > > > > > The vote will run for 5 weekdays, until Tuesday, July 13 at 3:50 > AM > > > > IST. > > > > > > > > > > > > > > > > > > I have done a few testing with my pseudo cluster. My +1 to start. > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > Brahma Reddy Battula > > > > > > > > > > > > > > > > > > 1. project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in > > > (3.3.0) > > > > > AND > > > > > > fixVersion not in (3.2.0, 3.2.1, 3.1.3) AND status = Resolved > ORDER > > > BY > > > > > > fixVersion ASC > > > > > > > > > > > >
