[jira] [Commented] (YARN-5139) [Umbrella] Move YARN scheduler towards global scheduler

2017-02-06 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855496#comment-15855496 ] Tao Yang commented on YARN-5139: Hi, [~leftnoteasy]. YARN-5716 seems not to be able to lookup multiple

[jira] [Commented] (YARN-6092) Refreshing CapacityScheduler page throws NPE

2017-02-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874229#comment-15874229 ] Tao Yang commented on YARN-6092: Hi,[~ajithshetty]. I found this problem when doing test on hadoop-2.9, and

[jira] [Commented] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-03-01 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891423#comment-15891423 ] Tao Yang commented on YARN-6259: Hi, [~rohithsharma]. Thank you for looking into this issue. {quote} I am

[jira] [Created] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-02-28 Thread Tao Yang (JIRA)
Tao Yang created YARN-6257: -- Summary: CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key Key: YARN-6257 URL: https://issues.apache.org/jira/browse/YARN-6257

[jira] [Updated] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-02-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6257: --- Description: In response string of CapacityScheduler REST API, scheduler/schedulerInfo/health/operationsInfo

[jira] [Updated] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-02-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6257: --- Attachment: YARN-6257.001.patch Attach a patch for review. > CapacityScheduler REST API produces incorrect

[jira] [Commented] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-03-01 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889796#comment-15889796 ] Tao Yang commented on YARN-6257: Hi, [~sunilg]. Thank you for looking into the issue. {quote} It will break

[jira] [Updated] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-03-01 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6257: --- Description: In response string of CapacityScheduler REST API, scheduler/schedulerInfo/health/operationsInfo

[jira] [Created] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-03-01 Thread Tao Yang (JIRA)
Tao Yang created YARN-6259: -- Summary: Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices Key: YARN-6259 URL: https://issues.apache.org/jira/browse/YARN-6259

[jira] [Updated] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-03-01 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6259: --- Attachment: YARN-6259.001.patch Attach a patch for review > Support pagination and optimize data transfer

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Commented] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531569#comment-15531569 ] Tao Yang commented on YARN-5683: Thanks for comments, [~grey]. We think

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Created] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
Tao Yang created YARN-5683: -- Summary: Support specifying storage type for per-application local dirs Key: YARN-5683 URL: https://issues.apache.org/jira/browse/YARN-5683 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Attachment: flow_diagram_for_MapReduce.png > Support specifying storage type for per-application local dirs >

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Attachment: YARN-5683-1.patch > Support specifying storage type for per-application local dirs >

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-09-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h1. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Updated] (YARN-5749) Fail to localize resources after health status for local dirs changed

2016-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5749: --- Summary: Fail to localize resources after health status for local dirs changed (was: Fail to localize

[jira] [Created] (YARN-5749) Fail to localize resources after health status for local dirs changed occurred by the change of FileContext#setUMask

2016-10-19 Thread Tao Yang (JIRA)
Tao Yang created YARN-5749: -- Summary: Fail to localize resources after health status for local dirs changed occurred by the change of FileContext#setUMask Key: YARN-5749 URL:

[jira] [Updated] (YARN-5749) Fail to localize resources after health status for local dirs changed

2016-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5749: --- Description: HADOOP-13440 updated FileContext#setUMask method to change umask from local variable to global

[jira] [Commented] (YARN-5749) Fail to localize resources after health status for local dirs changed

2016-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588266#comment-15588266 ] Tao Yang commented on YARN-5749: Contacted with [~ajisakaa]], This problem can be solved by the patch in

[jira] [Resolved] (YARN-5749) Fail to localize resources after health status for local dirs changed

2016-10-19 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang resolved YARN-5749. Resolution: Duplicate > Fail to localize resources after health status for local dirs changed >

[jira] [Commented] (YARN-5683) Support specifying storage type for per-application local dirs

2016-10-11 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565030#comment-15565030 ] Tao Yang commented on YARN-5683: Add YARN-5683-2.patch and update the description. Updates: (1) Fix

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-10-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Attachment: YARN-5683-3.patch > Support specifying storage type for per-application local dirs >

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-10-11 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Attachment: YARN-5683-2.patch flow_diagram_for_MapReduce-2.png > Support specifying storage

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2016-10-11 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Updated] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6029: --- Priority: Critical (was: Blocker) > CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called

[jira] [Updated] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6029: --- Attachment: YARN-6029.002.patch > CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784388#comment-15784388 ] Tao Yang commented on YARN-6029: Thanks [~wangda]. Updated priority to Critical and Attached new patch for

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786683#comment-15786683 ] Tao Yang commented on YARN-6029: Thanks [~wangda] for correcting me. I missed to realize that

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786715#comment-15786715 ] Tao Yang commented on YARN-6029: Patch updated. Update Assignee to [~leftnoteasy] since I have made little

[jira] [Updated] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6029: --- Assignee: Wangda Tan (was: Tao Yang) > CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is

[jira] [Updated] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6029: --- Attachment: YARN-6029-branch-2.8.001.patch > CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo

[jira] [Created] (YARN-6044) Resource bar of Capacity Scheduler UI does not show correctly

2016-12-29 Thread Tao Yang (JIRA)
Tao Yang created YARN-6044: -- Summary: Resource bar of Capacity Scheduler UI does not show correctly Key: YARN-6044 URL: https://issues.apache.org/jira/browse/YARN-6044 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2017-01-02 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794011#comment-15794011 ] Tao Yang commented on YARN-6029: Hi,[~Naganarasimha] I thought the application could be finished in another

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782067#comment-15782067 ] Tao Yang commented on YARN-6029: Thanks [~gtCarrera9] for correcting me. There is something wrong in my

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781969#comment-15781969 ] Tao Yang commented on YARN-6029: Thanks [~Naganarasimha] [~djp] [~leftnoteasy] for your suggestions.

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782578#comment-15782578 ] Tao Yang commented on YARN-6029: Thanks [~wangda] & [~Naganarasimha] ! {quote} Agree but IIUC based on 2.8

[jira] [Updated] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-27 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6029: --- Attachment: YARN-6029.001.patch deadlock.jstack > CapacityScheduler deadlock when

[jira] [Created] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved co

2016-12-27 Thread Tao Yang (JIRA)
Tao Yang created YARN-6029: -- Summary: CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved container Key: YARN-6029

[jira] [Created] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-28 Thread Tao Yang (JIRA)
Tao Yang created YARN-6403: -- Summary: Invalid local resource request can raise NPE and make NM exit Key: YARN-6403 URL: https://issues.apache.org/jira/browse/YARN-6403 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6403: --- Description: Recently we found this problem on our testing environment. The app that caused this problem

[jira] [Updated] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6403: --- Attachment: YARN-6403.002.patch [~jlowe] Thanks for correcting me. The last server-side change is not proper

[jira] [Commented] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-28 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946419#comment-15946419 ] Tao Yang commented on YARN-6403: [~Naganarasimha] Yes, I would like to work on this and will submit a patch

[jira] [Updated] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-29 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6403: --- Attachment: YARN-6403.001.patch Attach a patch for review. * Add local resources check in

[jira] [Updated] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-04-03 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6403: --- Attachment: YARN-6403.004.patch YARN-6403.branch-2.8.004.patch Thanks [~jlowe] for your

[jira] [Commented] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-04-05 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958249#comment-15958249 ] Tao Yang commented on YARN-6403: [~jlowe], thanks for review and committing! > Invalid local resource

[jira] [Commented] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950419#comment-15950419 ] Tao Yang commented on YARN-6403: [~jlowe] Thanks for your time! {quote} I believe it's appropriate to

[jira] [Updated] (YARN-6403) Invalid local resource request can raise NPE and make NM exit

2017-03-31 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6403: --- Attachment: YARN-6403.branch-2.8.003.patch Attach new patch for branch-2.8 > Invalid local resource request

[jira] [Updated] (YARN-5683) Support specifying storage type for per-application local dirs

2017-08-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-5683: --- Description: h3. Introduction * Some applications of various frameworks (Flink, Spark and MapReduce etc)

[jira] [Resolved] (YARN-6044) Resource bar of Capacity Scheduler UI does not show correctly

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang resolved YARN-6044. Resolution: Duplicate > Resource bar of Capacity Scheduler UI does not show correctly >

[jira] [Commented] (YARN-6044) Resource bar of Capacity Scheduler UI does not show correctly

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124767#comment-16124767 ] Tao Yang commented on YARN-6044: Thanks [~djp] and [~sunilg] for your reply. The solution makes sense to

[jira] [Created] (YARN-7003) DRAINING state of queues can't be recovered after RM restart

2017-08-12 Thread Tao Yang (JIRA)
Tao Yang created YARN-7003: -- Summary: DRAINING state of queues can't be recovered after RM restart Key: YARN-7003 URL: https://issues.apache.org/jira/browse/YARN-7003 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-7003) DRAINING state of queues can't be recovered after RM restart

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7003: --- Affects Version/s: 2.9.0 > DRAINING state of queues can't be recovered after RM restart >

[jira] [Updated] (YARN-7003) DRAINING state of queues can't be recovered after RM restart

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7003: --- Affects Version/s: (was: 3.0.0-alpha3) 3.0.0-alpha4 > DRAINING state of queues

[jira] [Commented] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124563#comment-16124563 ] Tao Yang commented on YARN-6257: [~leftnoteasy], thanks for the reply. Yes, duplicated keys in JSON object

[jira] [Commented] (YARN-6629) NPE occurred when container allocation proposal is applied but its resource requests are removed before

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124569#comment-16124569 ] Tao Yang commented on YARN-6629: Sorry for the late reply. Thanks [~sunilg] for reviewing this issue. Yes,

[jira] [Updated] (YARN-6629) NPE occurred when container allocation proposal is applied but its resource requests are removed before

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6629: --- Attachment: YARN-6629.002.patch Uploaded a new patch with test case. > NPE occurred when container allocation

[jira] [Created] (YARN-7004) Add configs cache to optimize refreshQueues performance for large scale queues

2017-08-12 Thread Tao Yang (JIRA)
Tao Yang created YARN-7004: -- Summary: Add configs cache to optimize refreshQueues performance for large scale queues Key: YARN-7004 URL: https://issues.apache.org/jira/browse/YARN-7004 Project: Hadoop YARN

[jira] [Updated] (YARN-7004) Add configs cache to optimize refreshQueues performance for large scale queues

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7004: --- Attachment: YARN-7004.001.patch Uploaded v1 patch for review. > Add configs cache to optimize refreshQueues

[jira] [Updated] (YARN-7003) DRAINING state of queues can't be recovered after RM restart

2017-08-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7003: --- Attachment: YARN-7003.001.patch > DRAINING state of queues can't be recovered after RM restart >

[jira] [Created] (YARN-7005) Skip unnecessary sorting and iterating process for child queues without pending resource to optimize schedule performance

2017-08-13 Thread Tao Yang (JIRA)
Tao Yang created YARN-7005: -- Summary: Skip unnecessary sorting and iterating process for child queues without pending resource to optimize schedule performance Key: YARN-7005 URL:

[jira] [Updated] (YARN-7004) Add configs cache to optimize refreshQueues performance for large scale of queues

2017-08-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7004: --- Summary: Add configs cache to optimize refreshQueues performance for large scale of queues (was: Add configs

[jira] [Commented] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-08-16 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129955#comment-16129955 ] Tao Yang commented on YARN-6257: Thanks [~sunilg] and [~leftnoteasy]. it makes sense to me. I will update

[jira] [Updated] (YARN-7037) Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-17 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7037: --- Attachment: YARN-7037.branch-2.8.001.patch Upload v1 patch for review. > Optimize data transfer with

[jira] [Created] (YARN-7037) Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-17 Thread Tao Yang (JIRA)
Tao Yang created YARN-7037: -- Summary: Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices Key: YARN-7037 URL: https://issues.apache.org/jira/browse/YARN-7037 Project:

[jira] [Commented] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-17 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130303#comment-16130303 ] Tao Yang commented on YARN-6259: Thanks [~djp] for your suggestions. It makes sense to me. I have created

[jira] [Commented] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-08-10 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122755#comment-16122755 ] Tao Yang commented on YARN-6257: This problem was imported by YARN-3293 (2.8.0+). The operationsInfo can't

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.branch-2.003.patch Upload a patch for branch-2. Thanks [~sunilg] for review and

[jira] [Commented] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-07-12 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083824#comment-16083824 ] Tao Yang commented on YARN-6678: I confirmed that it's fine. Thanks [~sunilg] for your help. > Committer

[jira] [Comment Edited] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087164#comment-16087164 ] Tao Yang edited comment on YARN-6714 at 7/14/17 2:14 PM: - Sorry to have misplaced

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: (was: YARN-6714.branch-2.005.patch) > IllegalStateException while handling APP_ATTEMPT_REMOVED

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: YARN-6678.005.patch Sure, upload new patch to resolve the conflict with YARN-6714 in

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-13 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.branch-2.004.patch For the check javac warning, It seems that the custom generic types

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.branch-2.006.patch > IllegalStateException while handling APP_ATTEMPT_REMOVED event when

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.branch-2.005.patch > IllegalStateException while handling APP_ATTEMPT_REMOVED event when

[jira] [Updated] (YARN-6714) IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-07-14 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.branch-2.005.patch Sorry to have misplaced the actual types. Upload a new patch. >

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-07-25 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: YARN-6678.branch-2.005.patch Attached branch-2 patch for cleanly applying. Thanks [~sunilg] and

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: YARN-6678.004.patch Thanks [~sunilg] for your time. As your mentioned, This new patch adds

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: (was: YARN-6678.004.patch) > Committer thread crashes with IllegalStateException in

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-30 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: YARN-6678.004.patch > Committer thread crashes with IllegalStateException in async-scheduling mode

[jira] [Updated] (YARN-7037) Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-23 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7037: --- Attachment: YARN-7037.001.patch YARN-7037.branch-2.8.001.patch Upload v1 patch for trunk and

[jira] [Updated] (YARN-7037) Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

2017-08-23 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-7037: --- Attachment: (was: YARN-7037.branch-2.8.001.patch) > Optimize data transfer with zero-copy approach for

[jira] [Updated] (YARN-6257) CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key

2017-08-18 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6257: --- Attachment: YARN-6257.002.patch Upload v2 patch for review. RM REST document has been updated. >

[jira] [Comment Edited] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055413#comment-16055413 ] Tao Yang edited comment on YARN-6714 at 6/20/17 9:47 AM: - Thanks [~leftnoteasy] for

[jira] [Comment Edited] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055413#comment-16055413 ] Tao Yang edited comment on YARN-6714 at 6/20/17 9:38 AM: - Thanks [~leftnoteasy] for

[jira] [Commented] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055310#comment-16055310 ] Tao Yang commented on YARN-6678: Thanks [~leftnoteasy] for your comments. {quote} Instead of using

[jira] [Commented] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055413#comment-16055413 ] Tao Yang commented on YARN-6714: Thanks [~leftnoteasy] for reviewing the patch. {quote} Could you move test

[jira] [Comment Edited] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055413#comment-16055413 ] Tao Yang edited comment on YARN-6714 at 6/20/17 9:08 AM: - Thanks [~leftnoteasy] for

[jira] [Updated] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6678: --- Attachment: YARN-6678.003.patch Updated the patch without adding new method to CapacityScheduler. Thanks

[jira] [Updated] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.002.patch Updated the patch with moving test case to

[jira] [Comment Edited] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060372#comment-16060372 ] Tao Yang edited comment on YARN-6678 at 6/23/17 4:11 AM: - Thanks [~sunilg] for your

[jira] [Commented] (YARN-6678) Committer thread crashes with IllegalStateException in async-scheduling mode of CapacityScheduler

2017-06-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060372#comment-16060372 ] Tao Yang commented on YARN-6678: Thanks [~sunilg] for your comments. {quote} 1. In FiCaSchedulerApp#accept,

[jira] [Updated] (YARN-6714) RM crashed with IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler

2017-06-23 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6714: --- Attachment: YARN-6714.003.patch Update the patch with adding comments for sanity check of attemptId. Thanks

[jira] [Created] (YARN-6737) Rename getApplicationAttempt to getCurrentAttempt in AbstractYarnScheduler/CapacityScheduler

2017-06-23 Thread Tao Yang (JIRA)
Tao Yang created YARN-6737: -- Summary: Rename getApplicationAttempt to getCurrentAttempt in AbstractYarnScheduler/CapacityScheduler Key: YARN-6737 URL: https://issues.apache.org/jira/browse/YARN-6737

[jira] [Updated] (YARN-6629) NPE occurred when container allocation proposal is applied but its resource requests are removed before

2017-05-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6629: --- Description: I wrote a test case to reproduce another problem for branch-2 and found new NPE error, log:

[jira] [Created] (YARN-6629) NPE occurred when container allocation proposal is applied but its resource requests are removed before

2017-05-22 Thread Tao Yang (JIRA)
Tao Yang created YARN-6629: -- Summary: NPE occurred when container allocation proposal is applied but its resource requests are removed before Key: YARN-6629 URL: https://issues.apache.org/jira/browse/YARN-6629

[jira] [Updated] (YARN-6629) NPE occurred when container allocation proposal is applied but its resource requests are removed before

2017-05-22 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-6629: --- Attachment: YARN-6629.001.patch Attach a patch for review. > NPE occurred when container allocation proposal

  1   2   3   4   5   6   7   8   9   10   >