[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302759#comment-17302759 ] Haibo Chen commented on YARN-1151: -- cherry-picked to branch-2.10  > Ability to configure auxiliary

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1151: - Fix Version/s: 2.10.2 > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Updated] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2.10

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10698: -- Attachment: YARN-10698.branch-2.10.00.patch > Backport YARN-1151 (load auxiliary service from HDFS

[jira] [Updated] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2.10

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10698: -- Target Version/s: 2.10.2 > Backport YARN-1151 (load auxiliary service from HDFS archives) to

[jira] [Updated] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2.10

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10698: -- Affects Version/s: 2.10.1 > Backport YARN-1151 (load auxiliary service from HDFS archives) to

[jira] [Updated] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2.10

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10698: -- Summary: Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2.10 (was: Backport

[jira] [Updated] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2

2021-03-16 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10698: -- Target Version/s: (was: 2.10.2) > Backport YARN-1151 (load auxiliary service from HDFS archives) to

[jira] [Created] (YARN-10698) Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2

2021-03-16 Thread Haibo Chen (Jira)
Haibo Chen created YARN-10698: - Summary: Backport YARN-1151 (load auxiliary service from HDFS archives) to branch-2 Key: YARN-10698 URL: https://issues.apache.org/jira/browse/YARN-10698 Project: Hadoop

[jira] [Commented] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-25 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291263#comment-17291263 ] Haibo Chen commented on YARN-10651: --- I updated the patch to add some logging.  Unit test wise, the key

[jira] [Updated] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-25 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10651: -- Attachment: YARN-10651.01.patch > CapacityScheduler crashed with NPE in >

[jira] [Updated] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10651: -- Attachment: YARN-10651.00.patch > CapacityScheduler crashed with NPE in >

[jira] [Updated] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10651: -- Affects Version/s: 2.10.0 2.10.1 > CapacityScheduler crashed with NPE in >

[jira] [Commented] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290641#comment-17290641 ] Haibo Chen commented on YARN-10651: --- When a node update scheduler event is processed by the scheduler

[jira] [Updated] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10651: -- Attachment: event_seq.jpg > CapacityScheduler crashed with NPE in >

[jira] [Commented] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290636#comment-17290636 ] Haibo Chen commented on YARN-10651: --- Relevant RM log   {code:java} 6553854:2021-02-24 17:06:33,934

[jira] [Created] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
Haibo Chen created YARN-10651: - Summary: CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource() Key: YARN-10651 URL: https://issues.apache.org/jira/browse/YARN-10651 Project:

[jira] [Updated] (YARN-10651) CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource()

2021-02-24 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10651: -- Description: {code:java} 2021-02-24 17:07:39,798 FATAL org.apache.hadoop.yarn.event.EventDispatcher:

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-28 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.02.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-28 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.branch-2.10.03.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Commented] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-28 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1738#comment-1738 ] Haibo Chen commented on YARN-10467: --- Thanks for catching this, [~Jim_Brennan]! I'll quickly update the

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Fix Version/s: (was: 2.10.2) > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Affects Version/s: 3.2.1 > ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers >

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Affects Version/s: 3.0.3 3.1.4 > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.branch-2.10.02.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Commented] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221766#comment-17221766 ] Haibo Chen commented on YARN-10467: --- Thanks for the review, [~jhung]. I updated the branch-2.10 patch

[jira] [Commented] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221561#comment-17221561 ] Haibo Chen commented on YARN-10467: --- Updated the patches to address checkstyle issues. The unit test

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.branch-2.10.01.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.01.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-26 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.00.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Updated] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-26 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-10467: -- Attachment: YARN-10467.branch-2.10.00.patch > ContainerIdPBImpl objects can be leaked in

[jira] [Created] (YARN-10467) ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers

2020-10-19 Thread Haibo Chen (Jira)
Haibo Chen created YARN-10467: - Summary: ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers Key: YARN-10467 URL: https://issues.apache.org/jira/browse/YARN-10467 Project: Hadoop

[jira] [Commented] (YARN-8213) Add Capacity Scheduler performance metrics

2020-03-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069102#comment-17069102 ] Haibo Chen commented on YARN-8213: -- Thanks for the backport, [~jhung] . +1 on the 2.10 patch. > Add

[jira] [Commented] (YARN-10200) Add number of containers to RMAppManager summary

2020-03-23 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065116#comment-17065116 ] Haibo Chen commented on YARN-10200: --- +1 on 001 patch pending the checkstyle fix. > Add number of

[jira] [Commented] (YARN-10039) Allow disabling app submission from REST endpoints

2019-12-18 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999400#comment-16999400 ] Haibo Chen commented on YARN-10039: --- I see. +1 on the patch. > Allow disabling app submission from

[jira] [Commented] (YARN-10039) Allow disabling app submission from REST endpoints

2019-12-17 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-10039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998668#comment-16998668 ] Haibo Chen commented on YARN-10039: --- [~jhung] Shall we disable all REST endpoints that update/change

[jira] [Commented] (YARN-9730) Support forcing configured partitions to be exclusive based on app node label

2019-09-23 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936262#comment-16936262 ] Haibo Chen commented on YARN-9730: -- I see. Thanks for the clarification, [~jhung]. There are some minor

[jira] [Commented] (YARN-9824) Fall back to configured queue ordering policy class name

2019-09-10 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926885#comment-16926885 ] Haibo Chen commented on YARN-9824: -- 001 Patch looks good to me. +1  from me. > Fall back to configured

[jira] [Commented] (YARN-9806) TestNMSimulator#testNMSimulator fails in branch-2

2019-09-03 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921616#comment-16921616 ] Haibo Chen commented on YARN-9806: -- +1 on the patch. Thanks for the fix, [~jhung]! >

[jira] [Commented] (YARN-9730) Support forcing configured partitions to be exclusive based on app node label

2019-08-28 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918149#comment-16918149 ] Haibo Chen commented on YARN-9730: -- Thanks for the patch [~jhung]!  Trying to understand the

[jira] [Commented] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability

2019-08-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917058#comment-16917058 ] Haibo Chen commented on YARN-9770: -- Thanks [~jhung] for the patch. The patch looks good to me overall. I

[jira] [Commented] (YARN-9438) launchTime not written to state store for running applications

2019-08-27 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917051#comment-16917051 ] Haibo Chen commented on YARN-9438: -- +1 on the latest 004 patch > launchTime not written to state store

[jira] [Commented] (YARN-9438) launchTime not written to state store for running applications

2019-08-20 Thread Haibo Chen (Jira)
[ https://issues.apache.org/jira/browse/YARN-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911679#comment-16911679 ] Haibo Chen commented on YARN-9438: -- Thanks for the patch, [~jhung]. I have one question on the patch.

[jira] [Updated] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic

2019-08-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9559: - Fix Version/s: 3.3.0 > Create AbstractContainersLauncher for pluggable ContainersLauncher logic >

[jira] [Commented] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic

2019-08-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901450#comment-16901450 ] Haibo Chen commented on YARN-9559: -- The unit test failure is reported at  YARN-5857, independent of the

[jira] [Commented] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic

2019-08-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901373#comment-16901373 ] Haibo Chen commented on YARN-9559: -- +1 on the latest patch pending Jenkins. > Create

[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889074#comment-16889074 ] Haibo Chen commented on YARN-9668: -- +1 on the latest branch-2 patch. > UGI conf doesn't read user

[jira] [Commented] (YARN-9646) DistributedShell tests failed to bind to a local host name

2019-07-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886578#comment-16886578 ] Haibo Chen commented on YARN-9646: -- [~HappyRay] I have fixed the minor import issues along with my

[jira] [Updated] (YARN-9646) DistributedShell tests failed to bind to a local host name

2019-07-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9646: - Summary: DistributedShell tests failed to bind to a local host name (was: Yarn miniYarn cluster tests

[jira] [Commented] (YARN-9646) Yarn miniYarn cluster tests failed to bind to a local host name

2019-07-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885353#comment-16885353 ] Haibo Chen commented on YARN-9646: -- Thanks [~ste...@apache.org] for the clarification. Agreed that the

[jira] [Updated] (YARN-9646) Yarn miniYarn cluster tests failed to bind to a local host name

2019-07-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9646: - Attachment: YARN-9646.00.patch > Yarn miniYarn cluster tests failed to bind to a local host name >

[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883374#comment-16883374 ] Haibo Chen commented on YARN-9668: -- [~jhung] Do you intend to fix this for branch-2 as well?The

[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883326#comment-16883326 ] Haibo Chen commented on YARN-9668: -- +1 on the latest patch. Committing it shortly. > UGI conf doesn't

[jira] [Commented] (YARN-9646) Yarn miniYarn cluster tests failed to bind to a local host name

2019-07-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881455#comment-16881455 ] Haibo Chen commented on YARN-9646: -- [~ste...@apache.org] Do you still have any concerns? > Yarn miniYarn

[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880717#comment-16880717 ] Haibo Chen commented on YARN-9668: -- +1 pending Jenkins. > UGI conf doesn't read user overridden

[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup

2019-07-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880618#comment-16880618 ] Haibo Chen commented on YARN-9668: -- Thanks for the patch, [~jhung]. I have a minor comment. Can we use

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2019-05-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839715#comment-16839715 ] Haibo Chen commented on YARN-2194: -- +1 on the patch pending Jenkins. > Cgroups cease to work in RHEL7 >

[jira] [Commented] (YARN-9529) Log correct cpu controller path on error while initializing CGroups.

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834122#comment-16834122 ] Haibo Chen commented on YARN-9529: -- +1 on the patch. Committed to trunk, branch-3.2/1/0.x and branch-2.

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Affects Version/s: 3.2.0 > Log correct cpu controller path on error >

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Labels: 2.10.0 3.3.0 (was: 2.10.0) > Log correct cpu controller path on error >

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error while initializing CGroups.

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Summary: Log correct cpu controller path on error while initializing CGroups. (was: Log correct cpu

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Component/s: nodemanager > Log correct cpu controller path on error >

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Affects Version/s: 2.10.0 > Log correct cpu controller path on error >

[jira] [Updated] (YARN-9529) Log correct cpu controller path on error

2019-05-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9529: - Labels: 2.10.0 (was: ) > Log correct cpu controller path on error >

[jira] [Created] (YARN-9111) NM crashes because Fair scheduler promotes a container that has not been pulled by AM

2018-12-11 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-9111: Summary: NM crashes because Fair scheduler promotes a container that has not been pulled by AM Key: YARN-9111 URL: https://issues.apache.org/jira/browse/YARN-9111 Project:

[jira] [Created] (YARN-9110) Fair Scheduler promotion does not update container execution type when the application is killed

2018-12-11 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-9110: Summary: Fair Scheduler promotion does not update container execution type when the application is killed Key: YARN-9110 URL: https://issues.apache.org/jira/browse/YARN-9110

[jira] [Commented] (YARN-9008) Extend YARN distributed shell with file localization feature

2018-12-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718096#comment-16718096 ] Haibo Chen commented on YARN-9008: -- +1 on the latest patch. Checking it in shortly. > Extend YARN

[jira] [Commented] (YARN-9051) Integrate multiple CustomResourceTypesConfigurationProvider implementations into one

2018-12-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717858#comment-16717858 ] Haibo Chen commented on YARN-9051: -- The unit test failures and license issue are unrelated. +1 on the

[jira] [Commented] (YARN-9008) Extend YARN distributed shell with file localization feature

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715459#comment-16715459 ] Haibo Chen commented on YARN-9008: -- {quote}I took that from {{Client.java}}, where it's also called

[jira] [Updated] (YARN-9087) Improve logging for initialization of Resource plugins

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9087: - Fix Version/s: 3.3.0 > Improve logging for initialization of Resource plugins >

[jira] [Commented] (YARN-9087) Better logging for initialization of Resource plugins

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715401#comment-16715401 ] Haibo Chen commented on YARN-9087: -- +1 on the latest patch. > Better logging for initialization of

[jira] [Updated] (YARN-9087) Improve logging for initialization of Resource plugins

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9087: - Summary: Improve logging for initialization of Resource plugins (was: Better logging for initialization

[jira] [Commented] (YARN-8738) FairScheduler configures maxResources or minResources as negative, the value parse to a positive number.

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715375#comment-16715375 ] Haibo Chen commented on YARN-8738: -- +1 on the latest patch. Checking it in shortly. > FairScheduler

[jira] [Updated] (YARN-8738) FairScheduler should not parse negative maxResources or minResources values as positive

2018-12-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8738: - Summary: FairScheduler should not parse negative maxResources or minResources values as positive (was:

[jira] [Commented] (YARN-9008) Extend YARN distributed shell with file localization feature

2018-12-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712143#comment-16712143 ] Haibo Chen commented on YARN-9008: -- Thanks [~pbacsko] for the patch!  A few minor comments 1) We are

[jira] [Commented] (YARN-9087) Better logging for initialization of Resource plugins

2018-12-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712108#comment-16712108 ] Haibo Chen commented on YARN-9087: -- Thanks [~snemeth] for the patch. A few nits about the patch. 1) Not

[jira] [Commented] (YARN-8738) FairScheduler configures maxResources or minResources as negative, the value parse to a positive number.

2018-12-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712039#comment-16712039 ] Haibo Chen commented on YARN-8738: -- Thanks [~snemeth] for the elaboration. What I meant is that because

[jira] [Commented] (YARN-9035) Allow better troubleshooting of FS container assignments and lack of container assignments

2018-12-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711943#comment-16711943 ] Haibo Chen commented on YARN-9035: -- Thanks [~snemeth] for the patch. This would be very helpful when it

[jira] [Commented] (YARN-9035) Allow better troubleshooting of FS container assignments and lack of container assignments

2018-12-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711944#comment-16711944 ] Haibo Chen commented on YARN-9035: -- [~wilfreds] probably has a much better idea of what is preferred from

[jira] [Commented] (YARN-9025) Make TestFairScheduler#testChildMaxResources more reliable, as it is flaky now

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710803#comment-16710803 ] Haibo Chen commented on YARN-9025: -- +1 on the latest patch. Checking it in shortly. > Make

[jira] [Assigned] (YARN-9025) TestFairScheduler#testChildMaxResources is flaky

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-9025: Assignee: Szilard Nemeth (was: Haibo Chen) > TestFairScheduler#testChildMaxResources is flaky >

[jira] [Assigned] (YARN-9025) TestFairScheduler#testChildMaxResources is flaky

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-9025: Assignee: Haibo Chen (was: Szilard Nemeth) > TestFairScheduler#testChildMaxResources is flaky >

[jira] [Updated] (YARN-9025) TestFairScheduler#testChildMaxResources is flaky

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9025: - Summary: TestFairScheduler#testChildMaxResources is flaky (was: Make

[jira] [Commented] (YARN-9051) Integrate multiple CustomResourceTypesConfigurationProvider implementations into one

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710796#comment-16710796 ] Haibo Chen commented on YARN-9051: -- Thanks [~snemeth] for the patch. Looks good overall. Some nits: 1)

[jira] [Commented] (YARN-9019) Ratio calculation of ResourceCalculator implementations could return NaN

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710731#comment-16710731 ] Haibo Chen commented on YARN-9019: -- +1 checking in shortly > Ratio calculation of ResourceCalculator

[jira] [Updated] (YARN-8985) Improve debug log in FSParentQueue when assigning container

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8985: - Summary: Improve debug log in FSParentQueue when assigning container (was: FSParentQueue: debug log

[jira] [Commented] (YARN-8985) FSParentQueue: debug log missing when assigning container

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710723#comment-16710723 ] Haibo Chen commented on YARN-8985: -- +1 and checking it in shortly. > FSParentQueue: debug log missing

[jira] [Commented] (YARN-8738) FairScheduler configures maxResources or minResources as negative, the value parse to a positive number.

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710720#comment-16710720 ] Haibo Chen commented on YARN-8738: -- Thanks [~snemeth] for the patch. Two nits:  1) we can replace

[jira] [Updated] (YARN-8994) Fix race condition between move app and queue cleanup in Fair Scheduler

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8994: - Summary: Fix race condition between move app and queue cleanup in Fair Scheduler (was: Fix for race

[jira] [Commented] (YARN-8994) Fix for race condition in move app and queue cleanup in FS

2018-12-05 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710701#comment-16710701 ] Haibo Chen commented on YARN-8994: -- +1 on the patch. Committing to trunk shortly. > Fix for race

[jira] [Commented] (YARN-9066) Deprecate Fair Scheduler min share

2018-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704353#comment-16704353 ] Haibo Chen commented on YARN-9066: -- Go for it. > Deprecate Fair Scheduler min share >

[jira] [Created] (YARN-9066) Depreciate Fair Scheduler min share

2018-11-27 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-9066: Summary: Depreciate Fair Scheduler min share Key: YARN-9066 URL: https://issues.apache.org/jira/browse/YARN-9066 Project: Hadoop YARN Issue Type: Improvement

[jira] [Updated] (YARN-9066) Depreciate Fair Scheduler min share

2018-11-27 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-9066: - Attachment: Proposal_Deprecate_FS_Min_Share.pdf > Depreciate Fair Scheduler min share >

[jira] [Commented] (YARN-9025) Make TestFairScheduler#testChildMaxResources more reliable, as it is flaky now

2018-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693863#comment-16693863 ] Haibo Chen commented on YARN-9025: -- [~snemeth] I notice the existence of MockRM.drainEvents(). Do that

[jira] [Commented] (YARN-8992) Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue

2018-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693855#comment-16693855 ] Haibo Chen commented on YARN-8992: -- +1. Checking it in shortly > Fair scheduler can delete a dynamic

[jira] [Resolved] (YARN-9026) DefaultOOMHandler should mark preempted containers as killed

2018-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-9026. -- Resolution: Invalid > DefaultOOMHandler should mark preempted containers as killed >

[jira] [Commented] (YARN-9026) DefaultOOMHandler should mark preempted containers as killed

2018-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693638#comment-16693638 ] Haibo Chen commented on YARN-9026: -- Ah, I was not aware of this.  Indeed, this is already taken care of

[jira] [Created] (YARN-9026) DefaultOOMHandler should mark preempted containers as killed

2018-11-15 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-9026: Summary: DefaultOOMHandler should mark preempted containers as killed Key: YARN-9026 URL: https://issues.apache.org/jira/browse/YARN-9026 Project: Hadoop YARN

[jira] [Created] (YARN-8992) Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue

2018-11-08 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-8992: Summary: Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue Key: YARN-8992 URL: https://issues.apache.org/jira/browse/YARN-8992

[jira] [Updated] (YARN-8990) Fix fair scheduler race condition in app submit and queue cleanup

2018-11-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8990: - Summary: Fix fair scheduler race condition in app submit and queue cleanup (was: FS: race condition in

[jira] [Commented] (YARN-8992) Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue

2018-11-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680630#comment-16680630 ] Haibo Chen commented on YARN-8992: -- See [^YARN-8990.002.patch] for the proposed fix included in the

  1   2   3   4   5   6   7   8   9   10   >