Re: [VOTE] Moving Ozone to a separated Apache project

2020-09-25 Thread Weiwei Yang
+1 Weiwei On Friday, September 25, 2020, Dinesh Chitlangia wrote: > +1 > > Thanks, > Dinesh > > On Fri, Sep 25, 2020 at 2:00 AM Elek, Marton wrote: > > > Hi all, > > > > Thank you for all the feedback and requests, > > > > As we discussed in the previous thread(s) [1], Ozone is proposed to be

Re: [ANNOUNCE] New Apache Hadoop Committer - He Xiaoqiao

2020-06-11 Thread Weiwei Yang
Congratulations Xiaoqiao! On Thu, Jun 11, 2020 at 11:20 AM Sree Vaddi wrote: > Congratulations, He Xiaoqiao. > > Thank you./Sree > > > > On Thursday, June 11, 2020, 9:54:32 AM PDT, Chao Sun < > sunc...@apache.org> wrote: > > Congrats Xiaoqiao! > > On Thu, Jun 11, 2020 at 9:36 AM Ayush

Re: [DISCUSS] making Ozone a separate Apache project

2020-05-13 Thread Weiwei Yang
+1 Thanks, Marton for starting the discussion. One question, for the committers who contributed to Ozone before and got the committer-role in the past (like me), will they carry the committer-role to the new repo? On Wed, May 13, 2020 at 9:07 AM Dinesh Chitlangia wrote: > +1 > Thank you Marton

[jira] [Resolved] (YARN-9050) [Umbrella] Usability improvements for scheduler activities

2020-03-13 Thread Weiwei Yang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-9050. --- Hadoop Flags: Reviewed Resolution: Fixed > [Umbrella] Usability improvements for schedu

Re: [Announcement] DiDi HDFS successfully upgraded from 2.7.2 to 3.2.0

2019-12-09 Thread Weiwei Yang
Awesome! You guys rock!! Any plan to upgrade YARN to latest version too? -- Weiwei On Dec 9, 2019, 5:20 PM -0800, Hui Fei , wrote: > Hi Eric, > > It's Rolling Upgrade without downtime. We have a cluster with thousands of > nodes and 5 namespace, it takes 1 month to do the upgrade. Upgrading >

Re: Submarine spin-off proposal approved by Apache Board

2019-10-16 Thread Weiwei Yang
Congratulations to all the submariners !!! Thanks Weiwei On Oct 16, 2019, 3:42 PM -0700, Wangda Tan , wrote: > Hi Devs, > > I'm happy to let everybody know that Submarine spin-off proposal finally > got approved by Apache board. We will work with Apache board to finish the > rest of spin-off

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-19 Thread Weiwei Yang
+1 (binding) - Downloaded source, setup a single node cluster - Verified basic HDFS operations, put/get/cat etc - Verified basic YARN restful apis, cluster/nodes/scheduler, all seem good - Run several distributed shell job Thanks Weiwei On Sep 19, 2019, 4:28 PM +0800, Sunil Govindan , wrote: >

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Weiwei Yang
+1 (binding) Thanks Weiwei On Wed, Sep 18, 2019 at 6:35 AM Wangda Tan wrote: > +1 (binding). > > From my experiences of Submarine project, I think moving to a separate repo > helps. > > - Wangda > > On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan wrote: > > > +1 (binding). > > > > IIUC, there

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

2019-08-26 Thread Weiwei Yang
+1 Thanks Weiwei On Aug 27, 2019, 9:40 AM +0800, Xun Liu , wrote: > +1 (non-binding) > This is the best news. > > Peter Bacsko 于2019年8月27日周二 上午4:59写道: > > > +1 (non-binding) > > > > On Sat, Aug 24, 2019 at 4:06 AM Wangda Tan wrote: > > > > > Hi devs, > > > > > > This is a voting thread to move

[jira] [Created] (YARN-9698) [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler

2019-07-25 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9698: - Summary: [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler Key: YARN-9698 URL: https://issues.apache.org/jira/browse/YARN-9698 Project: Hadoop

Re: Aug Hadoop Community Meetup in China

2019-07-23 Thread Weiwei Yang
Hi Junping Thanks. I would like to get a slot to talk about our new open source project: YuniKorn. Thanks Weiwei On Jul 23, 2019, 5:08 PM +0800, 俊平堵 , wrote: > Thanks for these positive feedbacks! The local community has voted the date > and location to be 8/10, Beijing. So please book your

Re: want to become a contributor

2019-07-21 Thread Weiwei Yang
Hi Runzhou Just added you as a contributor, you should be good to go now. Thanks Weiwei On Sun, Jul 21, 2019 at 4:10 PM babo wu wrote: > Hi, > I want to contribute to Hadoop YARN. > Would you please give me the contributor permission? > My JIRA is runzhouwu. > Thx. > > >

Re: Aug Hadoop Community Meetup in China

2019-07-18 Thread Weiwei Yang
Hi Junping Thanks for organizing this event ! Just finished the survey, looking forward to meet folks in the coming meet up. Thanks Weiwei On Jul 18, 2019, 3:28 PM +0800, 俊平堵 , wrote: > Hi, all! > > I am glad to let you know that we are organizing > Hadoop Contributors Meetup in China on Aug. >

Re: [VOTE] Force "squash and merge" option for PR merge on github UI

2019-07-17 Thread Weiwei Yang
Thanks Marton, +1 on this. Weiwei On Jul 17, 2019, 2:07 PM +0800, Elek, Marton , wrote: > Hi, > > Github UI (ui!) helps to merge Pull Requests to the proposed branch. > There are three different ways to do it [1]: > > 1. Keep all the different commits from the PR branch and create one >

[ANNOUNCE] New Apache Hadoop Committer - Tao Yang

2019-07-15 Thread Weiwei Yang
Hi Dear Apache Hadoop Community It's my pleasure to announce that Tao Yang has been elected as an Apache Hadoop committer, this is to recognize his contributions to Apache Hadoop YARN project. Congratulations and welcome on board! Weiwei (On behalf of the Apache Hadoop PMC)

[jira] [Resolved] (YARN-9655) AllocateResponse in FederationInterceptor lost applicationPriority

2019-07-01 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-9655. --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 > AllocateRespo

Re: Agenda & More Information about Hadoop Community Meetup @ Palo Alto, June 26

2019-06-25 Thread Weiwei Yang
Thanks Wangda. Will this event be recorded? It will be extremely helpful for people who are unable to join to catch up. Thanks Weiwei On Jun 26, 2019, 4:12 AM +0800, Wangda Tan , wrote: A friendly reminder, The meetup will take place tomorrow at 9:00 AM PDT to 4:00 PM PDT. The address is: 395

Re: [VOTE] Release Apache Hadoop Submarine 0.2.0 - RC0

2019-06-21 Thread Weiwei Yang
+1 (binding) Thanks Weiwei On Jun 21, 2019, 5:33 AM +0800, Wangda Tan , wrote: +1 Binding. Tested in local cluster and reviewed docs. Thanks! On Wed, Jun 19, 2019 at 3:20 AM Sunil Govindan wrote: +1 binding - tested in local cluster. - tried tony run time as well - doc seems fine now. -

[jira] [Created] (YARN-9441) Component name should start with Apache Hadoop for consistency

2019-04-04 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9441: - Summary: Component name should start with Apache Hadoop for consistency Key: YARN-9441 URL: https://issues.apache.org/jira/browse/YARN-9441 Project: Hadoop YARN

[jira] [Resolved] (YARN-7877) Node attributes error handling based on RM validation response

2019-03-20 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-7877. --- Resolution: Duplicate > Node attributes error handling based on RM validation respo

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

2019-02-04 Thread Weiwei Yang
+1 Weiwei -- Weiwei On Feb 5, 2019, 2:11 AM +0800, Steve Loughran , wrote: > +1, binding > > > On 1 Feb 2019, at 22:15, Wangda Tan wrote: > > > > Hi all, > > > > According to positive feedbacks from the thread [1] > > > > This is vote thread to start a new subproject named "hadoop-submarine" >

[jira] [Created] (YARN-9275) Add link to NodeAttributes doc in PlacementConstraints document

2019-02-02 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9275: - Summary: Add link to NodeAttributes doc in PlacementConstraints document Key: YARN-9275 URL: https://issues.apache.org/jira/browse/YARN-9275 Project: Hadoop YARN

[jira] [Resolved] (YARN-8731) Rich Placement constraints optimization and enhancements

2019-02-02 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-8731. --- Resolution: Duplicate Assignee: Weiwei Yang Close this issue and move all children to YARN-8766

[jira] [Resolved] (YARN-7746) Fix PlacementProcessor to support app priority

2019-02-02 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-7746. --- Resolution: Won't Do > Fix PlacementProcessor to support app prior

[jira] [Created] (YARN-9263) TestConfigurationNodeAttributesProvider fails after Mockito updated

2019-02-01 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9263: - Summary: TestConfigurationNodeAttributesProvider fails after Mockito updated Key: YARN-9263 URL: https://issues.apache.org/jira/browse/YARN-9263 Project: Hadoop YARN

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Weiwei Yang
Thanks for proposing this Wangda, my +1 as well. It is amazing to see the progress made in Submarine last year, the community grows fast and quiet collaborative. I can see the reasons to get it release faster in its own cycle. And at the same time, the Ozone way works very well. — Weiwei On Feb

Re: Testing Allocation Tag NameSpace

2019-01-30 Thread Weiwei Yang
Hi Prabhu I just did a quick look. From the scheduling request side, it should work. But the placement constraint parser in distributed shell did not parse the namespace, that looks like a missing piece. Do you want to open a ticket to track this? Thanks Weiwei Power of Open and

[jira] [Created] (YARN-9220) Add more UT coverage for RM nodes web page

2019-01-22 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9220: - Summary: Add more UT coverage for RM nodes web page Key: YARN-9220 URL: https://issues.apache.org/jira/browse/YARN-9220 Project: Hadoop YARN Issue Type

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-10 Thread Weiwei Yang
t; I have done few testing with my pseudo cluster. My +1 to start. > > > > > > > > Regards, > > > > Sunil > > > > > > > > [1] > > > > > https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E > > > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0) > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved > > ORDER BY fixVersion ASC > > > Wilfred Spiegelenburg | Software Engineer > cloudera.com <https://www.cloudera.com/> > > > > > > > > -- Weiwei Yang

[jira] [Created] (YARN-9186) [CSI] Upgrade CSI proto to v1.0

2019-01-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9186: - Summary: [CSI] Upgrade CSI proto to v1.0 Key: YARN-9186 URL: https://issues.apache.org/jira/browse/YARN-9186 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Resolved] (YARN-9178) TestRMAdminCli#testHelp is failing in trunk

2019-01-06 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-9178. --- Resolution: Duplicate Fix Version/s: (was: 3.3.0) > TestRMAdminCli#testHelp is fail

[jira] [Created] (YARN-9143) Aux service system-class is loading from a wrong property

2018-12-17 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9143: - Summary: Aux service system-class is loading from a wrong property Key: YARN-9143 URL: https://issues.apache.org/jira/browse/YARN-9143 Project: Hadoop YARN Issue

Re: [DISCUSS] Move to gitbox

2018-12-10 Thread Weiwei Yang
scribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > > ----- > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > -- Weiwei Yang

[jira] [Created] (YARN-9086) [CSI] Run csi-driver-adaptor as aux service

2018-12-06 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9086: - Summary: [CSI] Run csi-driver-adaptor as aux service Key: YARN-9086 URL: https://issues.apache.org/jira/browse/YARN-9086 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-9038) Add ability to publish volumes on node managers

2018-11-19 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9038: - Summary: Add ability to publish volumes on node managers Key: YARN-9038 URL: https://issues.apache.org/jira/browse/YARN-9038 Project: Hadoop YARN Issue Type: Sub

[jira] [Created] (YARN-9037) Ignore volume resource in resource calculators based on tags

2018-11-19 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-9037: - Summary: Ignore volume resource in resource calculators based on tags Key: YARN-9037 URL: https://issues.apache.org/jira/browse/YARN-9037 Project: Hadoop YARN

[jira] [Created] (YARN-8987) Usability improvements for adding centralised node-attributes from CLI

2018-11-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8987: - Summary: Usability improvements for adding centralised node-attributes from CLI Key: YARN-8987 URL: https://issues.apache.org/jira/browse/YARN-8987 Project: Hadoop YARN

[jira] [Created] (YARN-8988) Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled

2018-11-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8988: - Summary: Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled Key: YARN-8988 URL: https://issues.apache.org/jira/browse/YARN-8988

[jira] [Created] (YARN-8971) Improvement to TestContainerAllocation#testContainerRejectionWhenAskBeyondDynamicMax

2018-11-05 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8971: - Summary: Improvement to TestContainerAllocation#testContainerRejectionWhenAskBeyondDynamicMax Key: YARN-8971 URL: https://issues.apache.org/jira/browse/YARN-8971 Project

[jira] [Created] (YARN-8970) Improve the debug message in CS#allocateContainerOnSingleNode

2018-11-05 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8970: - Summary: Improve the debug message in CS#allocateContainerOnSingleNode Key: YARN-8970 URL: https://issues.apache.org/jira/browse/YARN-8970 Project: Hadoop YARN

[jira] [Created] (YARN-8953) Add CSI adaptor module

2018-10-29 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8953: - Summary: Add CSI adaptor module Key: YARN-8953 URL: https://issues.apache.org/jira/browse/YARN-8953 Project: Hadoop YARN Issue Type: Sub-task Reporter

[jira] [Created] (YARN-8940) Add volume as a top-level attribute in service spec

2018-10-23 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8940: - Summary: Add volume as a top-level attribute in service spec Key: YARN-8940 URL: https://issues.apache.org/jira/browse/YARN-8940 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8902) Add volume manager that manages CSI volume lifecycle

2018-10-17 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8902: - Summary: Add volume manager that manages CSI volume lifecycle Key: YARN-8902 URL: https://issues.apache.org/jira/browse/YARN-8902 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-16 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8896: - Summary: Limit the maximum number of container assignments per heartbeat Key: YARN-8896 URL: https://issues.apache.org/jira/browse/YARN-8896 Project: Hadoop YARN

[jira] [Created] (YARN-8877) Extend service spec to allow setting resource attributes

2018-10-15 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8877: - Summary: Extend service spec to allow setting resource attributes Key: YARN-8877 URL: https://issues.apache.org/jira/browse/YARN-8877 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8873) Add CSI java-based client library

2018-10-12 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8873: - Summary: Add CSI java-based client library Key: YARN-8873 URL: https://issues.apache.org/jira/browse/YARN-8873 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Resolved] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-10-09 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-8468. --- Resolution: Fixed > Enable the use of queue based maximum container allocation limit and > imp

[jira] [Created] (YARN-8836) Add tags and attributes in resource definition

2018-09-29 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8836: - Summary: Add tags and attributes in resource definition Key: YARN-8836 URL: https://issues.apache.org/jira/browse/YARN-8836 Project: Hadoop YARN Issue Type: Sub

[jira] [Created] (YARN-8811) Support Container Storage Interface (CSI) in YARN

2018-09-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8811: - Summary: Support Container Storage Interface (CSI) in YARN Key: YARN-8811 URL: https://issues.apache.org/jira/browse/YARN-8811 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8765) Extend YARN to support SET type of resources

2018-09-12 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8765: - Summary: Extend YARN to support SET type of resources Key: YARN-8765 URL: https://issues.apache.org/jira/browse/YARN-8765 Project: Hadoop YARN Issue Type: New

Re: [Discuss] Merge discussion for Node attribute support feature YARN-3409

2018-09-04 Thread Weiwei Yang
nfiguration > details. This feature works from end-to-end and we tested this in our local > cluster. Branch code is run against trunk and tracked via [4]. > > We would love to get your thoughts before opening a voting thread. > > Special thanks to a team of folks who worked hard and

Re: HADOOP-14163 proposal for new hadoop.apache.org

2018-09-02 Thread Weiwei Yang
>>> I got multiple valuable feedback and I improved the proposed site > > >>> according to the comments. Allen had some concerns about the used > > >>> technologies (hugo vs. mvn-site) and I answered all the questions > > why > > >>> I think mvn-site is the best for documentation and hugo is best > > for > > >>> generating site. > > >>> > > >>> > > >>> I would like to finish this effort/jira: I would like to start a > > >>> discussion about using this proposed version and approach as a > new > > >>> site of Apache Hadoop. Please let me know what you think. > > >>> > > >>> > > >>> Thanks a lot, > > >>> Marton > > >>> > > >>> > > - > > >>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > >>> For additional commands, e-mail: > common-dev-h...@hadoop.apache.org > > >>> > > >> > > >> > > >> > > - > > >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > >> For additional commands, e-mail: > common-dev-h...@hadoop.apache.org > > >> > > > > > > > - > > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > - > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > > > > > > -- Weiwei Yang

[jira] [Resolved] (YARN-7990) Node attribute prefix definition and validation

2018-08-31 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-7990. --- Resolution: Done Mark it as done for now. First version we simply support rm.yarn.io and nm.yarn.io

[jira] [Resolved] (YARN-7812) Improvements to Rich Placement Constraints in YARN

2018-08-29 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-7812. --- Resolution: Fixed Fix Version/s: 3.2.0 Thanks [~asuresh], [~kkaranasos], [~leftnoteasy

[jira] [Created] (YARN-8723) Typo in CS init error message when resource calculator is not correctly set

2018-08-28 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8723: - Summary: Typo in CS init error message when resource calculator is not correctly set Key: YARN-8723 URL: https://issues.apache.org/jira/browse/YARN-8723 Project: Hadoop

[jira] [Created] (YARN-8715) Make allocation tags in the placement spec optional for node-attributes

2018-08-25 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8715: - Summary: Make allocation tags in the placement spec optional for node-attributes Key: YARN-8715 URL: https://issues.apache.org/jira/browse/YARN-8715 Project: Hadoop YARN

[jira] [Resolved] (YARN-8704) Improve the error message for an invalid docker rw mount to be more informative

2018-08-23 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-8704. --- Resolution: Invalid > Improve the error message for an invalid docker rw mount to be m

[jira] [Closed] (YARN-8704) Improve the error message for an invalid docker rw mount to be more informative

2018-08-23 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang closed YARN-8704. - > Improve the error message for an invalid docker rw mount to be more > infor

[jira] [Created] (YARN-8704) Improve the error message for an invalid docker rw mount to be more informative

2018-08-23 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8704: - Summary: Improve the error message for an invalid docker rw mount to be more informative Key: YARN-8704 URL: https://issues.apache.org/jira/browse/YARN-8704 Project

[jira] [Created] (YARN-8576) Fix the broken anchor in SLS site-doc

2018-07-25 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8576: - Summary: Fix the broken anchor in SLS site-doc Key: YARN-8576 URL: https://issues.apache.org/jira/browse/YARN-8576 Project: Hadoop YARN Issue Type: Bug

[jira] [Resolved] (YARN-8559) Expose scheduling configuration info in Resource Manager's /conf endpoint

2018-07-20 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-8559. --- Resolution: Not A Problem Target Version/s: 3.1.0, 3.0.0 (was: 3.0.0, 3.1.0) > Exp

[jira] [Created] (YARN-8560) TestCapacityScheduler#testCSReservationWithRootUnblocked fails on branch-3.0

2018-07-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8560: - Summary: TestCapacityScheduler#testCSReservationWithRootUnblocked fails on branch-3.0 Key: YARN-8560 URL: https://issues.apache.org/jira/browse/YARN-8560 Project: Hadoop

[jira] [Created] (YARN-8557) Exclude lagged/unhealthy/decommissioned nodes in async allocating thread

2018-07-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8557: - Summary: Exclude lagged/unhealthy/decommissioned nodes in async allocating thread Key: YARN-8557 URL: https://issues.apache.org/jira/browse/YARN-8557 Project: Hadoop YARN

[jira] [Created] (YARN-8556) Support dynamic policy updates in Capacity Scheduler

2018-07-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8556: - Summary: Support dynamic policy updates in Capacity Scheduler Key: YARN-8556 URL: https://issues.apache.org/jira/browse/YARN-8556 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8555) Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options

2018-07-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8555: - Summary: Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options Key: YARN-8555 URL: https://issues.apache.org/jira/browse/YARN-8555

[jira] [Created] (YARN-8554) Add more E2E tests for placement constraints

2018-07-20 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8554: - Summary: Add more E2E tests for placement constraints Key: YARN-8554 URL: https://issues.apache.org/jira/browse/YARN-8554 Project: Hadoop YARN Issue Type: Sub

[jira] [Created] (YARN-8546) A reserved container might be released multiple times under async scheduling

2018-07-18 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8546: - Summary: A reserved container might be released multiple times under async scheduling Key: YARN-8546 URL: https://issues.apache.org/jira/browse/YARN-8546 Project: Hadoop

[jira] [Created] (YARN-8521) NPE in AllocationTagsManager when a container is removed more than once

2018-07-12 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8521: - Summary: NPE in AllocationTagsManager when a container is removed more than once Key: YARN-8521 URL: https://issues.apache.org/jira/browse/YARN-8521 Project: Hadoop YARN

[jira] [Resolved] (YARN-4192) Add YARN metric logging periodically to a seperate file

2018-07-11 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-4192. --- Resolution: Won't Fix I am closing this now, like [~aw] mentioned, this should be already handled

[jira] [Created] (YARN-8511) When AM releases a container, RM removes allocation tags before it is released by NM

2018-07-10 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8511: - Summary: When AM releases a container, RM removes allocation tags before it is released by NM Key: YARN-8511 URL: https://issues.apache.org/jira/browse/YARN-8511 Project

[jira] [Created] (YARN-8446) Support of managing multi-dimensional resources

2018-06-21 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8446: - Summary: Support of managing multi-dimensional resources Key: YARN-8446 URL: https://issues.apache.org/jira/browse/YARN-8446 Project: Hadoop YARN Issue Type: New

[jira] [Created] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-05 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8394: - Summary: Improve data locality documentation for Capacity Scheduler Key: YARN-8394 URL: https://issues.apache.org/jira/browse/YARN-8394 Project: Hadoop YARN Issue

[jira] [Resolved] (YARN-6289) Fail to achieve data locality when runing MapReduce and Spark on HDFS

2018-06-05 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-6289. --- Resolution: Duplicate Fix Version/s: 2.9.0 3.0.0 > Fail to achieve d

[jira] [Created] (YARN-8351) RM is flooded with node attributes manager logs

2018-05-23 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8351: - Summary: RM is flooded with node attributes manager logs Key: YARN-8351 URL: https://issues.apache.org/jira/browse/YARN-8351 Project: Hadoop YARN Issue Type: Sub

YARN-8320 CPU isolation for latency-sensitive services

2018-05-23 Thread Weiwei Yang
Hello everyone My colleague and I uploaded a proposal to support CPU isolation for latency-sensitive services with cpuset, the issue is https://issues.apache.org/jira/browse/YARN-8320, detail design doc attached. If you are interested on this, we are looking forward for your comments and

[jira] [Created] (YARN-8332) Incorrect min/max allocation property name in resource types doc

2018-05-21 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8332: - Summary: Incorrect min/max allocation property name in resource types doc Key: YARN-8332 URL: https://issues.apache.org/jira/browse/YARN-8332 Project: Hadoop YARN

[jira] [Created] (YARN-8288) Fix wrong number of table columns in Resource Model doc

2018-05-14 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8288: - Summary: Fix wrong number of table columns in Resource Model doc Key: YARN-8288 URL: https://issues.apache.org/jira/browse/YARN-8288 Project: Hadoop YARN Issue

Re: Apache Hadoop 3.1.1 release plan

2018-05-10 Thread Weiwei Yang
Hi Wangda I would propose to have https://issues.apache.org/jira/browse/YARN-8015 included in 3.1.1. Once this is done, we get both intra and inter placement constraint covered so users could start to explore this feature. Otherwise the functionality is pretty limited. It has been Patch

[jira] [Created] (YARN-8213) Add Capacity Scheduler metrics

2018-04-26 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8213: - Summary: Add Capacity Scheduler metrics Key: YARN-8213 URL: https://issues.apache.org/jira/browse/YARN-8213 Project: Hadoop YARN Issue Type: Improvement

[jira] [Created] (YARN-8212) Pending backlog for async allocation threads should be configurable

2018-04-26 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8212: - Summary: Pending backlog for async allocation threads should be configurable Key: YARN-8212 URL: https://issues.apache.org/jira/browse/YARN-8212 Project: Hadoop YARN

[jira] [Created] (YARN-8195) Multiple target allocation tags in placement constraint is not working as expected

2018-04-21 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8195: - Summary: Multiple target allocation tags in placement constraint is not working as expected Key: YARN-8195 URL: https://issues.apache.org/jira/browse/YARN-8195 Project

[jira] [Created] (YARN-8188) RM Nodes UI data table index for sorting column is messed up

2018-04-19 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8188: - Summary: RM Nodes UI data table index for sorting column is messed up Key: YARN-8188 URL: https://issues.apache.org/jira/browse/YARN-8188 Project: Hadoop YARN

[jira] [Created] (YARN-8165) Incorrect queue name logging in AbstractContainerAllocator

2018-04-15 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8165: - Summary: Incorrect queue name logging in AbstractContainerAllocator Key: YARN-8165 URL: https://issues.apache.org/jira/browse/YARN-8165 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8152) Add chart in SLS to illustrate the throughput of the scheduler

2018-04-11 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8152: - Summary: Add chart in SLS to illustrate the throughput of the scheduler Key: YARN-8152 URL: https://issues.apache.org/jira/browse/YARN-8152 Project: Hadoop YARN

[jira] [Resolved] (YARN-7821) Constraint satisfaction checker support for inter-app constraints

2018-04-11 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-7821. --- Resolution: Duplicate > Constraint satisfaction checker support for inter-app constrai

[jira] [Created] (YARN-8127) Resource leak when async scheduling is enabled

2018-04-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8127: - Summary: Resource leak when async scheduling is enabled Key: YARN-8127 URL: https://issues.apache.org/jira/browse/YARN-8127 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-8113) Add doc about allocation tag namespace and inter-app constraints

2018-04-03 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8113: - Summary: Add doc about allocation tag namespace and inter-app constraints Key: YARN-8113 URL: https://issues.apache.org/jira/browse/YARN-8113 Project: Hadoop YARN

[jira] [Created] (YARN-8112) Relax cardinality check for intra-app affinity placement constraints

2018-04-03 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8112: - Summary: Relax cardinality check for intra-app affinity placement constraints Key: YARN-8112 URL: https://issues.apache.org/jira/browse/YARN-8112 Project: Hadoop YARN

[jira] [Created] (YARN-8111) Allocation Tags namespace should not allow null value

2018-04-02 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8111: - Summary: Allocation Tags namespace should not allow null value Key: YARN-8111 URL: https://issues.apache.org/jira/browse/YARN-8111 Project: Hadoop YARN Issue Type

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-02 Thread Weiwei Yang
Hi Wangda +1 (non-binding) - Smoke tests with teragen/terasort/DS jobs - Various of metrics, UI displays validation, compatible tests - Tested GPU resource-type - Verified RM fail-over, app-recovery - Verified 2 threads async-scheduling - Enabled placement constraints, tested

[jira] [Created] (YARN-8101) Add UT to verify node-attributes in RM nodes rest API

2018-03-31 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8101: - Summary: Add UT to verify node-attributes in RM nodes rest API Key: YARN-8101 URL: https://issues.apache.org/jira/browse/YARN-8101 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8094) Support configuration based node attribute provider

2018-03-29 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8094: - Summary: Support configuration based node attribute provider Key: YARN-8094 URL: https://issues.apache.org/jira/browse/YARN-8094 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8092) Expose node-attributes info via RM nodes REST API

2018-03-29 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8092: - Summary: Expose node-attributes info via RM nodes REST API Key: YARN-8092 URL: https://issues.apache.org/jira/browse/YARN-8092 Project: Hadoop YARN Issue Type

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-29 Thread Weiwei Yang
Hi Wangda While testing the build, we found a bug https://issues.apache.org/jira/browse/YARN-8085, it might cause NPE during RM fail-over. I think that needs to be included in 3.1.0. We have run some other tests, and they look good so far. This includes - Some basic teragen/terasort/DS jobs -

[jira] [Created] (YARN-8078) TestDistributedShell#testDSShellWithoutDomainV2 fails on trunk

2018-03-27 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8078: - Summary: TestDistributedShell#testDSShellWithoutDomainV2 fails on trunk Key: YARN-8078 URL: https://issues.apache.org/jira/browse/YARN-8078 Project: Hadoop YARN

[jira] [Created] (YARN-8076) Support to specify application tags in distributed shell

2018-03-26 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8076: - Summary: Support to specify application tags in distributed shell Key: YARN-8076 URL: https://issues.apache.org/jira/browse/YARN-8076 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8015) Support allocation tag namespaces in AppPlacementAllocator

2018-03-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8015: - Summary: Support allocation tag namespaces in AppPlacementAllocator Key: YARN-8015 URL: https://issues.apache.org/jira/browse/YARN-8015 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8013) Support app-tag namespace for allocation tags

2018-03-08 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8013: - Summary: Support app-tag namespace for allocation tags Key: YARN-8013 URL: https://issues.apache.org/jira/browse/YARN-8013 Project: Hadoop YARN Issue Type: Sub

[jira] [Created] (YARN-8008) Admin command to manage global placement constraints

2018-03-07 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8008: - Summary: Admin command to manage global placement constraints Key: YARN-8008 URL: https://issues.apache.org/jira/browse/YARN-8008 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag

2018-03-05 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8002: - Summary: Support NOT_SELF and ALL namespace types for allocation tag Key: YARN-8002 URL: https://issues.apache.org/jira/browse/YARN-8002 Project: Hadoop YARN

  1   2   >