Re: Apache Hadoop 3.1.1 release plan
Hi all, I'm working on RC for 3.1.1 now, if you have any commits to 3.1.x line, please commit them to branch-3.1. If it is a blocker for 3.1.1, please let me know so we can decide if we want to roll another RC or not. Thanks, Wangda On Thu, Jul 19, 2018 at 2:47 PM Wangda Tan wrote: > Hi all, > > After several blockers of 3.1.1 landed, I think we're pretty close to a > clean 3.1.1 branch ready for RC. > > By far we have two blockers targeted 3.1.1 [1], and there're 420 tickets > have fix version = 3.1.1 [2] > > As we previously communicated for 3.1.1 release date (May 01), we have > delayed for more than two months, which I want to get 3.1.1 released as > soon as possible. I just cut branch-3.1.1 for blockers only. Branch-3.1 > will be open for all bug fixes. > > I'm going to create RC0 by end of tomorrow or the last blocker get > resolved, whichever is the later. Please let me know if there are any other > blockers need to get in 3.1.1. > > [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, > Critical) AND resolution = Unresolved AND "Target Version/s" = 3.1.1 ORDER > BY priority DESC > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.1) > ORDER BY priority DESC > > Thanks, > Wangda > > > On Thu, May 10, 2018 at 6:40 PM Weiwei Yang > wrote: > >> Hi Wangda >> >> I would propose to have https://issues.apache.org/jira/browse/YARN-8015 >> included >> in 3.1.1. >> >> Once this is done, we get both intra and inter placement constraint >> covered so users could start to explore this feature. Otherwise the >> functionality is pretty limited. It has been Patch Available for a while, I >> just promoted it targeting to 3.1.1. Hope that makes sense. >> >> Thanks! >> >> -- >> Weiwei >> >> On 11 May 2018, 9:02 AM +0800, Wangda Tan , wrote: >> >> Hi all, >> >> As we previously proposed RC time (May 1st), we want to release 3.1.1 >> sooner if possible. As of now, 3.1.1 has 187 fixes [1] on top of 3.1.0, >> and >> there're 10 open blockers/criticals which target to 3.1.1 [2]. I just >> posted comments to these open criticals/blockers ticket owners asking >> about >> statuses. >> >> If everybody agrees, I propose start code freeze of branch-3.1 from Sat >> PDT >> time this week, only blockers/criticals can be committed to branch-3.1. To >> avoid the burden of committers, I want to delay cutting branch-3.1.1 as >> late as possible. If you have any major/minors (For severe issues please >> update priorities) tickets want to go to 3.1.1, please reply this email >> thread and we can look at them and make a call together. >> >> Please feel free to share your comments and suggestions. >> >> Thanks, >> Wangda >> >> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce") >> AND status = Resolved AND fixVersion = 3.1.1 >> [2] project in (YARN, HADOOP, MAPREDUCE, "Hadoop Development Tools") AND >> priority in (Blocker, Critical) AND resolution = Unresolved AND "Target >> Version/s" = 3.1.1 ORDER BY priority DESC >> >> >> On Thu, May 10, 2018 at 5:48 PM, Wangda Tan wrote: >> >> Thanks Brahma/Sunil, >> >> For YARN-8265, it is a too big change for 3.1.1, I just removed 3.1.1 from >> target version. >> For YARN-8236, it is a severe issue and I think it is close to finish. >> >> >> >> On Thu, May 10, 2018 at 3:08 AM, Sunil G wrote: >> >> >> Thanks Brahma. >> Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236. >> >> - Sunil >> >> >> On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula < >> brahmareddy.batt...@huawei.com> wrote: >> >> Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better >> addition to 3.1 line release for improving quality. >> >> Looks only following two are pending which are in review state. Hope you >> are monitoring these two. >> >> https://issues.apache.org/jira/browse/YARN-8265 >> https://issues.apache.org/jira/browse/YARN-8236 >> >> >> >> Note : https://issues.apache.org/jira/browse/YARN-8247==> committed >> branch-3.1 >> >> >> -Original Message- >> From: Wangda Tan [mailto:wheele...@gmail.com] >> Sent: 19 April 2018 17:49 >> To: Hadoop Common ; >> mapreduce-dev@hadoop.apache.org; Hdfs-dev ; >> yarn-...@hadoop.apache.org >> Subject: Apache Hadoop 3.1.1 release plan >> >> Hi, All >> >> We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the >> quality of the release, we plan to release 3.1.1 at May 06. The focus of >> 3.1.1 will be fixing blockers / critical bugs and other enhancements. So >> far there are 100 JIRAs [1] have fix version marked to 3.1.1. >> >> We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day. >> >> Please feel free to share your insights. >> >> Thanks, >> Wangda Tan >> >> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop >> Map/Reduce") AND fixVersion = 3.1.1 >> >> >> >>
Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/545/ [Aug 1, 2018 11:39:10 AM] (aw) YETUS-46. releasedocmaker should support JIRA authentication [Aug 1, 2018 11:40:17 AM] (aw) YETUS-647. ssh-based git repos do not work correctly under docker [Jul 31, 2018 2:20:40 PM] (msingh) HDDS-279. DeleteBlocks command should not be sent for open containers. [Jul 31, 2018 4:36:34 PM] (inigoiri) YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek [Jul 31, 2018 6:32:40 PM] (haibochen) YARN-8605. TestDominantResourceFairnessPolicy.testModWhileSorting is [Jul 31, 2018 7:08:00 PM] (wangda) YARN-8418. App local logs could leaked if log aggregation fails to [Jul 31, 2018 10:01:02 PM] (eyang) YARN-8579. Recover NMToken of previous attempted component data. [Jul 31, 2018 11:26:09 PM] (bharat) HDDS-271. Create a block iterator to iterate blocks in a container. [Aug 1, 2018 1:44:49 AM] (fabbri) HDFS-13322 fuse dfs - uid persists when switching between ticket caches. [Aug 1, 2018 3:03:30 AM] (sunilg) YARN-8397. Potential thread leak in ActivitiesManager. Contributed by [Aug 1, 2018 3:33:00 AM] (msingh) HDDS-226. Client should update block length in OM while committing the [Aug 1, 2018 5:34:53 AM] (wangda) YARN-8522. Application fails with InvalidResourceRequestException. (Zian [Aug 1, 2018 6:47:18 AM] (sunilg) YARN-8606. Opportunistic scheduling does not work post RM failover. [Aug 1, 2018 8:57:54 AM] (sunilg) YARN-8595. [UI2] Container diagnostic information is missing from -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.cli.TestAclCLI hadoop.cli.TestAclCLIWithPosixAclInheritance hadoop.cli.TestCacheAdminCLI hadoop.cli.TestCryptoAdminCLI hadoop.cli.TestDeleteCLI hadoop.cli.TestErasureCodingCLI hadoop.cli.TestHDFSCLI hadoop.cli.TestXAttrCLI hadoop.fs.contract.hdfs.TestHDFSContractAppend hadoop.fs.contract.hdfs.TestHDFSContractConcat hadoop.fs.contract.hdfs.TestHDFSContractCreate hadoop.fs.contract.hdfs.TestHDFSContractDelete hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus hadoop.fs.contract.hdfs.TestHDFSContractMkdir hadoop.fs.contract.hdfs.TestHDFSContractOpen hadoop.fs.contract.hdfs.TestHDFSContractPathHandle hadoop.fs.contract.hdfs.TestHDFSContractRename hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory hadoop.fs.contract.hdfs.TestHDFSContractSeek hadoop.fs.contract.hdfs.TestHDFSContractSetTimes hadoop.fs.loadGenerator.TestLoadGenerator hadoop.fs.permission.TestStickyBit hadoop.fs.shell.TestHdfsTextCommand hadoop.fs.TestEnhancedByteBufferAccess hadoop.fs.TestFcHdfsCreateMkdir hadoop.fs.TestFcHdfsPermission hadoop.fs.TestFcHdfsSetUMask hadoop.fs.TestGlobPaths hadoop.fs.TestHDFSFileContextMainOperations hadoop.fs.TestHDFSMultipartUploader hadoop.fs.TestHdfsNativeCodeLoader hadoop.fs.TestResolveHdfsSymlink hadoop.fs.TestSWebHdfsFileContextMainOperations hadoop.fs.TestSymlinkHdfsDisable hadoop.fs.TestSymlinkHdfsFileContext hadoop.fs.TestSymlinkHdfsFileSystem hadoop.fs.TestUnbuffer hadoop.fs.TestUrlStreamHandler hadoop.fs.TestWebHdfsFileContextMainOperations hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot hadoop.fs.viewfs.TestViewFileSystemHdfs hadoop.fs.viewfs.TestViewFileSystemLinkFallback hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash hadoop.fs.viewfs.TestViewFileSystemWithAcls hadoop.fs.viewfs.TestViewFileSystemWithTruncate hadoop.fs.viewfs.TestViewFileSystemWithXAttrs hadoop.fs.viewfs.TestViewFsAtHdfsRoot hadoop.fs.viewfs.TestViewFsDefaultValue hadoop.fs.viewfs.TestViewFsFileStatusHdfs hadoop.fs.viewfs.TestViewFsHdfs hadoop.fs.viewfs.TestViewFsWithAcls hadoop.fs.viewfs.TestViewFsWithXAttrs hadoop.hdfs.client.impl.TestBlockReaderLocal hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy hadoop.hdfs.client.impl.TestBlockReaderRemote hadoop.hdfs.client.impl.TestClientBlockVerification hadoop.hdfs.crypto.TestHdfsCryptoStreams hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer hadoop.hdfs.qjournal.client.TestEpochsAreUnique hadoop.hdfs.qjournal.client.TestQJMWithFaults hadoop.hdfs.qjournal.client.TestQuorumJournalManager hadoop.hdfs.qjournal.server.TestJournal hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.qjournal.server
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/ [Jul 31, 2018 11:57:51 AM] (nanda) HDDS-305. Datanode StateContext#addContainerActionIfAbsent will add [Jul 31, 2018 2:20:40 PM] (msingh) HDDS-279. DeleteBlocks command should not be sent for open containers. [Jul 31, 2018 4:36:34 PM] (inigoiri) YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek [Jul 31, 2018 6:32:40 PM] (haibochen) YARN-8605. TestDominantResourceFairnessPolicy.testModWhileSorting is [Jul 31, 2018 7:08:00 PM] (wangda) YARN-8418. App local logs could leaked if log aggregation fails to [Jul 31, 2018 10:01:02 PM] (eyang) YARN-8579. Recover NMToken of previous attempted component data. [Jul 31, 2018 11:26:09 PM] (bharat) HDDS-271. Create a block iterator to iterate blocks in a container. -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.security.TestFixKerberosTicketOrder hadoop.hdfs.server.balancer.TestBalancer hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestMRTimelineEventHandling cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-checkstyle-root.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/whitespace-eol.txt [9.4M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-hdds_client.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_common.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [4.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/diff-javadoc-javadoc-root.txt [760K] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/855/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-cte
