[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068041#comment-14068041 ] Jian Fang commented on YARN-796: As Sandy pointed out, seems the scope becomes bigger and

[jira] [Commented] (YARN-671) Add an interface on the RM to move NMs into a maintenance state

[ https://issues.apache.org/jira/browse/YARN-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069124#comment-14069124 ] Jian Fang commented on YARN-671: YARN-796 is adding label support for NMs, could label based

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069175#comment-14069175 ] Jian Fang commented on YARN-796: * RM has list of valid labels. (hot reloadable) This

[jira] [Created] (YARN-2380) The normalizeRequests method in SchedulerUtils always resets the vCore to 1

Jian Fang created YARN-2380: --- Summary: The normalizeRequests method in SchedulerUtils always resets the vCore to 1 Key: YARN-2380 URL: https://issues.apache.org/jira/browse/YARN-2380 Project: Hadoop YARN

[jira] [Commented] (YARN-2380) The normalizeRequests method in SchedulerUtils always resets the vCore to 1

[ https://issues.apache.org/jira/browse/YARN-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083164#comment-14083164 ] Jian Fang commented on YARN-2380: - Seems DominantResourceCalculator sets vcore correctly.

[jira] [Created] (YARN-2416) InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time

Jian Fang created YARN-2416: --- Summary: InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time Key: YARN-2416 URL:

[jira] [Commented] (YARN-2416) InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time

[ https://issues.apache.org/jira/browse/YARN-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096250#comment-14096250 ] Jian Fang commented on YARN-2416: - Here is the log for the events and state transitions.

[jira] [Commented] (YARN-2416) InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time

[ https://issues.apache.org/jira/browse/YARN-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096256#comment-14096256 ] Jian Fang commented on YARN-2416: - Seems for some reason, the RPC call for

[jira] [Commented] (YARN-2380) The normalizeRequests method in SchedulerUtils always resets the vCore to 1

[ https://issues.apache.org/jira/browse/YARN-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097225#comment-14097225 ] Jian Fang commented on YARN-2380: - Any reason why you think DefaultResourceCalculator

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152137#comment-14152137 ] Jian Fang commented on YARN-1680: - Hi, any update on the fix? We saw quick some jobs failed

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152156#comment-14152156 ] Jian Fang commented on YARN-1680: - Thanks. Looking forward to your patch.

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153485#comment-14153485 ] Jian Fang commented on YARN-1198: - I tried to merge in YARN-1857.3.patch and then merge in

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155235#comment-14155235 ] Jian Fang commented on YARN-1680: - I may be wrong because I don't understand the logic

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155282#comment-14155282 ] Jian Fang commented on YARN-1680: - Also, seems the variable blackListedResources in

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155344#comment-14155344 ] Jian Fang commented on YARN-1680: - Is there any behavior change from hadoop one to hadoop

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155381#comment-14155381 ] Jian Fang commented on YARN-1680: - Thanks Craig for your clarification. Is the cluster

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158289#comment-14158289 ] Jian Fang commented on YARN-1198: - Craig, thanks for your effort. I have already merged in

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007698#comment-14007698 ] Jian Fang commented on YARN-796: I like to add a use case to this JIRA. In a cloud

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009893#comment-14009893 ] Jian Fang commented on YARN-796: Hi Bikas, I think it is better to have the node manager to

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061452#comment-14061452 ] Jian Fang commented on YARN-796: In our environment, most likely the label condition will be

[jira] [Created] (YARN-3130) Progress report log is incredibly excessive in application master

Jian Fang created YARN-3130: --- Summary: Progress report log is incredibly excessive in application master Key: YARN-3130 URL: https://issues.apache.org/jira/browse/YARN-3130 Project: Hadoop YARN

[jira] [Commented] (YARN-896) Roll up for long-lived services in YARN

[ https://issues.apache.org/jira/browse/YARN-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303785#comment-14303785 ] Jian Fang commented on YARN-896: I created a jira YARN-3130 for excessive progress report

[jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate long lived

[ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283147#comment-14283147 ] Jian Fang commented on YARN-1039: - The container request could specify what tags/labels it

[jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate long lived

[ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283074#comment-14283074 ] Jian Fang commented on YARN-1039: - The term long lived relies on resource manager to

[jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate long lived

[ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14284446#comment-14284446 ] Jian Fang commented on YARN-1039: - The duration concept comes with a good intention, but

[jira] [Commented] (YARN-1039) Add parameter for YARN resource requests to indicate long lived

[ https://issues.apache.org/jira/browse/YARN-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14284232#comment-14284232 ] Jian Fang commented on YARN-1039: - Thanks Steve for your clarification. Seems the long

[jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup

[ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386960#comment-14386960 ] Jian Fang commented on YARN-2923: - Thanks Naganarasimha for your quick update. I think

[jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup

[ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387065#comment-14387065 ] Jian Fang commented on YARN-2923: - That would require that our controller to monitor the

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387310#comment-14387310 ] Jian Fang commented on YARN-796: I took a look at

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382333#comment-14382333 ] Jian Fang commented on YARN-796: Come back to this issue again since I am trying to merge

[jira] [Commented] (YARN-2495) Allow admin specify labels from each NM (Distributed configuration)

[ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382299#comment-14382299 ] Jian Fang commented on YARN-2495: - In a cloud environment such as Amazon EMR, a hadoop

[jira] [Commented] (YARN-2495) Allow admin specify labels from each NM (Distributed configuration)

[ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382555#comment-14382555 ] Jian Fang commented on YARN-2495: - Great, thanks. Will try them. Allow admin specify

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382486#comment-14382486 ] Jian Fang commented on YARN-796: Thanks. Seems

[jira] [Commented] (YARN-2495) Allow admin specify labels from each NM (Distributed configuration)

[ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382496#comment-14382496 ] Jian Fang commented on YARN-2495: - On each EC2 instance, the metadata about that instance

[jira] [Commented] (YARN-2495) Allow admin specify labels from each NM (Distributed configuration)

[ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382501#comment-14382501 ] Jian Fang commented on YARN-2495: - BTW, I haven't gone through all the details of YARN-2492

[jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup

[ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384983#comment-14384983 ] Jian Fang commented on YARN-2923: - Hi Naganarasimha, could you please update your patch? I

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391719#comment-14391719 ] Jian Fang commented on YARN-796: JIRA MAPREDUCE-6304 has been created for this purpose.

[jira] [Created] (YARN-3490) Add an application decorator to ClientRMService

Jian Fang created YARN-3490: --- Summary: Add an application decorator to ClientRMService Key: YARN-3490 URL: https://issues.apache.org/jira/browse/YARN-3490 Project: Hadoop YARN Issue Type: New

[jira] [Commented] (YARN-2306) leak of reservation metrics (fair scheduler)

[ https://issues.apache.org/jira/browse/YARN-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496648#comment-14496648 ] Jian Fang commented on YARN-2306: - Could someone please tell me which JIRA has fixed this

[jira] [Commented] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494973#comment-14494973 ] Jian Fang commented on YARN-2497: - Any update for this? Thanks. Changes for fair

[jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup

[ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14396106#comment-14396106 ] Jian Fang commented on YARN-2923: - Thanks Naganarasimha for your reply. I think the

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391701#comment-14391701 ] Jian Fang commented on YARN-796: Thanks Wangda for your clarification. Unfortunately the