Re: Hadoop 3.2 Release Plan proposal
Hi Folks, We are now close to RC as other blocker issues are now merged to trunk and branch-3.2. Last 2 critical issues are closer to merge and will be committed in few hours. With this, I will be creating 3.2.0 branch today and will go ahead with RC related process. - Sunil On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender wrote: > Hello, were there any updates around the 3.2.0 RC timing? All I see in the > current blockers are related to the new Submarine subproject, wasn't sure > if that is what is holding things up. > > Cheers, > Jon > > On Tue, Oct 2, 2018 at 7:13 PM, Sunil G wrote: > >> Thanks Robert and Haibo for quickly correcting same. >> Sigh, I somehow missed one file while committing the change. Sorry for the >> trouble. >> >> - Sunil >> >> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter >> wrote: >> >> > Looks like there's two that weren't updated: >> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT" . -r >> > --include=pom.xml >> > ./hadoop-project/pom.xml: >> > 3.2.0-SNAPSHOT >> > ./pom.xml:3.2.0-SNAPSHOT >> > >> > I've just pushed in an addendum commit to fix those. >> > In the future, please make sure to do a sanity compile when updating >> poms. >> > >> > thanks >> > - Robert >> > >> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri >> >> > wrote: >> > >> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in the >> >> top-level pom.xml? >> >> >> >> >> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G wrote: >> >> >> >> > Hi All >> >> > >> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset trunk >> to >> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all necessary >> >> > patches are pulled into branch-3.2. >> >> > >> >> > Thank You >> >> > - Sunil >> >> > >> >> > >> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G wrote: >> >> > >> >> > > Hi All >> >> > > >> >> > > We are now down to the last Blocker and HADOOP-15407 is merged to >> >> trunk. >> >> > > Thanks for the support. >> >> > > >> >> > > *Plan for RC* >> >> > > 3.2 branch cut and reset trunk : *25th Tuesday* >> >> > > RC0 for 3.2: *28th Friday* >> >> > > >> >> > > Thank You >> >> > > Sunil >> >> > > >> >> > > >> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G wrote: >> >> > > >> >> > >> Hi All >> >> > >> >> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all of you >> for >> >> > >> helping in this. I am following up on these tickets, once its >> closed >> >> we >> >> > >> will cut the 3.2 branch. >> >> > >> >> >> > >> Thanks >> >> > >> Sunil Govindan >> >> > >> >> >> > >> >> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G >> wrote: >> >> > >> >> >> > >>> Hi All, >> >> > >>> >> >> > >>> Inline with the original 3.2 communication proposal dated 17th >> July >> >> > >>> 2018, I would like to provide more updates. >> >> > >>> >> >> > >>> We are approaching previously proposed code freeze date >> (September >> >> 14, >> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and point >> >> > existing >> >> > >>> trunk to 3.3 if there are no issues. >> >> > >>> >> >> > >>> *Current Release Plan:* >> >> > >>> Feature freeze date : all features to merge by September 7, 2018. >> >> > >>> Code freeze date : blockers/critical only, no improvements and >> >> > >>> blocker/critical bug-fixes September 14, 2018. >> >> > >>> Release date: September 28, 2018 >> >> > >>> >> >> > >>> If any critical/blocker tickets which are targeted to 3.2.0, we >> >> need to >> >> > >>> backport to 3.2 post branch cut. >> >> > >>> >> >> > >>> Here's an updated 3.2.0 feature status: >> >> > >>> >> >> > >>> 1. Merged & Completed features: >> >> > >>> >> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning >> >> > >>> workloads Initial cut. >> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity >> >> > >>> Scheduler. >> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN >> Service >> >> > API >> >> > >>> and CLI. >> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. >> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation. Improvement >> >> works. >> >> > >>> >> >> > >>> 2. Features close to finish: >> >> > >>> >> >> > >>> - (Steve) S3Guard Phase III. Close to commit. >> >> > >>> - (Steve) S3a phase V. Close to commit. >> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit. >> >> > >>> >> >> > >>> 3. Tentative/Cancelled features for 3.2: >> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps >> from >> >> > >>> ATSv2. Patch in progress. >> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks >> >> challenging to >> >> > >>> be done before Aug 2018. >> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN applications. >> >> > >>> Challenging as more discussions are on-going. >> >> > >>> >> >> > >>> *Summary of 3.2.0 issues status:* >> >> > >>> 19 Blocker and Critical issues [1] are open, I am
[jira] [Created] (YARN-8878) Remove StringBuffer from ManagedParentQueue.java
BELUGA BEHR created YARN-8878:
-
Summary: Remove StringBuffer from ManagedParentQueue.java
Key: YARN-8878
URL: https://issues.apache.org/jira/browse/YARN-8878
Project: Hadoop YARN
Issue Type: Improvement
Components: resourcemanager
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
Attachments: YARN-8878.1.patch
Remove all {{StringBuffer}} references from the class {{ManagedParentQueue}}.
{{StringBuffer}} are synchronized and should be instead replaced with the
non-synchronized {{StringBuilder}}. However, in this case, just use {{SLF4J}}
parameter logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/ [Oct 14, 2018 2:15:01 AM] (jitendra) HDDS-579. ContainerStateMachine should fail subsequent transactions per [Oct 14, 2018 3:22:02 AM] (xiao) HADOOP-15849. Upgrade netty version to 3.10.6. [Oct 14, 2018 9:30:18 AM] (elek) HDDS-519. Implement ListBucket REST endpoint. Contributed by LiXin Ge. -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.TestEncryptionZonesWithKMS hadoop.hdfs.TestEncryptionZones hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-compile-javac-root.txt [300K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-patch-pylint.txt [40K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-hdds_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_common.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/diff-javadoc-javadoc-root.txt [752K] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt [116K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/927/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [312K]
[jira] [Created] (YARN-8877) Extend service spec to allow setting resource attributes
Weiwei Yang created YARN-8877: - Summary: Extend service spec to allow setting resource attributes Key: YARN-8877 URL: https://issues.apache.org/jira/browse/YARN-8877 Project: Hadoop YARN Issue Type: Sub-task Reporter: Weiwei Yang Extend yarn native service spec to support setting resource attributes in the spec file. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8876) [Submarine] After training, the monitoring program need close PS service
Xun Liu created YARN-8876: - Summary: [Submarine] After training, the monitoring program need close PS service Key: YARN-8876 URL: https://issues.apache.org/jira/browse/YARN-8876 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xun Liu Assignee: Xun Liu TensorFlow model after training, TensorFlow can not automatically turn off the PS service, so it needs to be actively shut down by the service status monitoring program. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
