[jira] [Updated] (YARN-2465) Make YARN unit tests work when pseudo distributed cluster is running

[ https://issues.apache.org/jira/browse/YARN-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2465: -- Attachment: YARN-2465.patch Most of the cases have been addressed in MiniYarnCluster by using port 0 so that

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046.patch In the patch, the container asks NodeStatusUpdater to send out of band heartbeats

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: (was: YARN-2046.patch) Out of band heartbeats are sent only on container kill and possibly too

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046.patch Out of band heartbeats are sent only on container kill and possibly too early

[jira] [Commented] (YARN-90) NodeManager should identify failed disks becoming good back again

[ https://issues.apache.org/jira/browse/YARN-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154211#comment-14154211 ] Ming Ma commented on YARN-90: - Thanks, Varun, Jason. Couple comments: 1. What if a dir is

[jira] [Commented] (YARN-90) NodeManager should identify failed disks becoming good back again

[ https://issues.apache.org/jira/browse/YARN-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158643#comment-14158643 ] Ming Ma commented on YARN-90: - Thanks, Varun. The main question about UNHEALTHY state is whether

[jira] [Created] (YARN-2649) Flaky test TestAMRMRPCNodeUpdates

Ming Ma created YARN-2649: - Summary: Flaky test TestAMRMRPCNodeUpdates Key: YARN-2649 URL: https://issues.apache.org/jira/browse/YARN-2649 Project: Hadoop YARN Issue Type: Bug Reporter:

[jira] [Updated] (YARN-2649) Flaky test TestAMRMRPCNodeUpdates

[ https://issues.apache.org/jira/browse/YARN-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2649: -- Attachment: YARN-2649.patch Fix the test code to wait until RMAppAttemptImpl gets to

[jira] [Updated] (YARN-2649) Flaky test TestAMRMRPCNodeUpdates

[ https://issues.apache.org/jira/browse/YARN-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2649: -- Attachment: YARN-2649-2.patch [~jianhe], thanks for the info. The latest patch should be able to address other

[jira] [Commented] (YARN-90) NodeManager should identify failed disks becoming good back again

[ https://issues.apache.org/jira/browse/YARN-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172050#comment-14172050 ] Ming Ma commented on YARN-90: - Thanks Varun. You and Jason discussed about disk clean up

[jira] [Commented] (YARN-90) NodeManager should identify failed disks becoming good back again

[ https://issues.apache.org/jira/browse/YARN-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175175#comment-14175175 ] Ming Ma commented on YARN-90: - Thanks Varun.The latest patch LGTM. NodeManager should identify

[jira] [Created] (YARN-2714) Localizer thread might stuck if NM is OOM

Ming Ma created YARN-2714: - Summary: Localizer thread might stuck if NM is OOM Key: YARN-2714 URL: https://issues.apache.org/jira/browse/YARN-2714 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-2714) Localizer thread might stuck if NM is OOM

[ https://issues.apache.org/jira/browse/YARN-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180042#comment-14180042 ] Ming Ma commented on YARN-2714: --- Thanks Zhihai for the information. Yes, setting the RPC

[jira] [Commented] (YARN-2578) NM does not failover timely if RM node network connection fails

[ https://issues.apache.org/jira/browse/YARN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180144#comment-14180144 ] Ming Ma commented on YARN-2578: --- Yeah, it is more than just * - RM, it could be * - NM and *

[jira] [Commented] (YARN-1756) capture the time when newApplication is called in RM

[ https://issues.apache.org/jira/browse/YARN-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13916686#comment-13916686 ] Ming Ma commented on YARN-1756: --- In between newApplication and submitApplication, MR client

[jira] [Commented] (YARN-1756) capture the time when newApplication is called in RM

[ https://issues.apache.org/jira/browse/YARN-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13916727#comment-13916727 ] Ming Ma commented on YARN-1756: --- That is right. capture the time when newApplication is

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13916958#comment-13916958 ] Ming Ma commented on YARN-221: -- Thanks, Jason. To fix the racing between container's exit by

[jira] [Updated] (YARN-1751) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

[ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1751: -- Attachment: YARN-1751.patch Here is the patch. Improve MiniYarnCluster and LogCLIHelpers for log aggregation

[jira] [Updated] (YARN-1751) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

[ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1751: -- Attachment: (was: YARN-1751.patch) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

[jira] [Commented] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13920507#comment-13920507 ] Ming Ma commented on YARN-445: -- Any comments on the proposal to do signaling via client - RM -

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Attachment: MRTasks.png MRJob.png YARNContainers.png Thanks Xuan. Let us

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Issue Type: Task (was: Sub-task) Parent: (was: YARN-386) Ability to signal containers

[jira] [Commented] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924701#comment-13924701 ] Ming Ma commented on YARN-445: -- Thanks all for the comments. I will create subtasks and feel

[jira] [Created] (YARN-1803) Signal container support in nodemanager

Ming Ma created YARN-1803: - Summary: Signal container support in nodemanager Key: YARN-1803 URL: https://issues.apache.org/jira/browse/YARN-1803 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-1804) Signal container request delivery from client to resourcemanager

Ming Ma created YARN-1804: - Summary: Signal container request delivery from client to resourcemanager Key: YARN-1804 URL: https://issues.apache.org/jira/browse/YARN-1804 Project: Hadoop YARN Issue

[jira] [Created] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

Ming Ma created YARN-1805: - Summary: Signal container request delivery from resourcemanager to nodemanager Key: YARN-1805 URL: https://issues.apache.org/jira/browse/YARN-1805 Project: Hadoop YARN

[jira] [Created] (YARN-1806) webUI update to allow end users to request thread dump

Ming Ma created YARN-1806: - Summary: webUI update to allow end users to request thread dump Key: YARN-1806 URL: https://issues.apache.org/jira/browse/YARN-1806 Project: Hadoop YARN Issue Type:

[jira] [Created] (YARN-1807) CLI update to allow people to signal a specfic container

Ming Ma created YARN-1807: - Summary: CLI update to allow people to signal a specfic container Key: YARN-1807 URL: https://issues.apache.org/jira/browse/YARN-1807 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Attachment: YARN-445-signal-container-via-rm.patch For discussion purpose, here is the patch for reference based

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-trunk-v2.patch Here is the patch to support log aggregation sampling at yarn layer. Yarn

[jira] [Commented] (YARN-85) Allow per job log aggregation configuration

[ https://issues.apache.org/jira/browse/YARN-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955601#comment-13955601 ] Ming Ma commented on YARN-85: - Regarding Seth's comment of container exit status is not

[jira] [Created] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

Ming Ma created YARN-1897: - Summary: Define SignalContainerRequest and SignalContainerResponse Key: YARN-1897 URL: https://issues.apache.org/jira/browse/YARN-1897 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962221#comment-13962221 ] Ming Ma commented on YARN-1897: --- +1 for SignalContainerRequest. For SignalContainerResponse,

[jira] [Commented] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968544#comment-13968544 ] Ming Ma commented on YARN-1897: --- Sounds good. How about IsCMDSubmissionCompleted? Define

[jira] [Commented] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976310#comment-13976310 ] Ming Ma commented on YARN-1897: --- Thanks, Xuan. I will merge this one with the version I have

[jira] [Commented] (YARN-1354) Recover applications upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978983#comment-13978983 ] Ming Ma commented on YARN-1354: --- Yeah, the FINISH_APP message lost can be addressed by having

[jira] [Commented] (YARN-1354) Recover applications upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979150#comment-13979150 ] Ming Ma commented on YARN-1354: --- Sorry, for FINISH_APP message lost scenario, NM can't get it

[jira] [Commented] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980343#comment-13980343 ] Ming Ma commented on YARN-1897: --- Looks good. Thanks, Xuan. SignalContainerResponse still has

[jira] [Reopened] (YARN-738) TestClientRMTokens is failing irregularly while running all yarn tests

[ https://issues.apache.org/jira/browse/YARN-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma reopened YARN-738: -- Just get the repro. 1. testShortCircuitRenewCancel uses InetAddress.getLocalHost(), which sometimes will return

[jira] [Updated] (YARN-738) TestClientRMTokens is failing irregularly while running all yarn tests

[ https://issues.apache.org/jira/browse/YARN-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-738: - Attachment: YARN-738.patch The patch fixes the test code to use NetUtils.createSocketAddr to create sockets from

[jira] [Assigned] (YARN-1751) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

[ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma reassigned YARN-1751: - Assignee: Ming Ma Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

[jira] [Commented] (YARN-1987) Wrapper for leveldb DBIterator to aid in handling database exceptions

[ https://issues.apache.org/jira/browse/YARN-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985894#comment-13985894 ] Ming Ma commented on YARN-1987: --- Jason, 1. LeveldbIterator.close rethrows IOException

[jira] [Commented] (YARN-1987) Wrapper for leveldb DBIterator to aid in handling database exceptions

[ https://issues.apache.org/jira/browse/YARN-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987059#comment-13987059 ] Ming Ma commented on YARN-1987: --- Jason, thanks for the clarification. The patch looks good to

[jira] [Updated] (YARN-2018) TestClientRMService.testTokenRenewalWrongUser fails occasionally

[ https://issues.apache.org/jira/browse/YARN-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2018: -- Attachment: YARN-2018.patch It seems HADOOP-10562 modified the message. Here is the patch.

[jira] [Updated] (YARN-1803) Signal container support in nodemanager

[ https://issues.apache.org/jira/browse/YARN-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1803: -- Attachment: YARN-1803.patch Here is the patch to support signal container functionality in node manager. 1.

[jira] [Assigned] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

[ https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma reassigned YARN-1805: - Assignee: Ming Ma Signal container request delivery from resourcemanager to nodemanager

[jira] [Updated] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager

[ https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1805: -- Attachment: YARN-1805.patch The patch includes YARN-1803 and YARN-1897 for jenkins to build. Signal container

[jira] [Commented] (YARN-2001) Persist NMs info for RM restart

[ https://issues.apache.org/jira/browse/YARN-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990215#comment-13990215 ] Ming Ma commented on YARN-2001: --- 1. In the HA set up, could we make standby RM hot by having

[jira] [Updated] (YARN-2050) Fix LogCLIHelpers to create the correct FileContext

[ https://issues.apache.org/jira/browse/YARN-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2050: -- Attachment: YARN-2050.patch Papreduce CLI and yarn CLI pass the configuration to LogCLIHelpers. LogCLIHelpers

[jira] [Updated] (YARN-1751) Improve MiniYarnCluster for log aggregation testing

[ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1751: -- Attachment: YARN-1751.patch Thanks, Jason. Here is the patch for MiniYarnCluster. I have opened

[jira] [Created] (YARN-2050) Fix LogCLIHelpers to create the correct FileContext

Ming Ma created YARN-2050: - Summary: Fix LogCLIHelpers to create the correct FileContext Key: YARN-2050 URL: https://issues.apache.org/jira/browse/YARN-2050 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-1751) Improve MiniYarnCluster for log aggregation testing

[ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1751: -- Summary: Improve MiniYarnCluster for log aggregation testing (was: Improve MiniYarnCluster and LogCLIHelpers

[jira] [Commented] (YARN-1803) Signal container support in nodemanager

[ https://issues.apache.org/jira/browse/YARN-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997062#comment-13997062 ] Ming Ma commented on YARN-1803: --- Vinod, thanks for the great feedback. So to summarize it,

[jira] [Updated] (YARN-2050) Fix LogCLIHelpers to create the correct FileContext

[ https://issues.apache.org/jira/browse/YARN-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2050: -- Attachment: YARN-2050-2.patch Thanks, Jason. You are right. remoteAppLogDir could point to a different type of

[jira] [Commented] (YARN-1352) Recover LogAggregationService upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002797#comment-14002797 ] Ming Ma commented on YARN-1352: --- Jason, not sure you will cover NonAggregatingLogHandler in a

[jira] [Updated] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Attachment: YARN-1897-4.patch Updated patch per Vinod's suggestions. 1. Clean up SignalContainerCommand. 2.

[jira] [Commented] (YARN-1803) Signal container support in nodemanager

[ https://issues.apache.org/jira/browse/YARN-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002847#comment-14002847 ] Ming Ma commented on YARN-1803: --- Vinod, I have updated YARN-1897. Please let me know if you

[jira] [Commented] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004166#comment-14004166 ] Ming Ma commented on YARN-1897: --- Chatted with Gera offline. The definition of

[jira] [Created] (YARN-2082) Support for alternative log aggregation mechanism

Ming Ma created YARN-2082: - Summary: Support for alternative log aggregation mechanism Key: YARN-2082 URL: https://issues.apache.org/jira/browse/YARN-2082 Project: Hadoop YARN Issue Type: New

[jira] [Commented] (YARN-2082) Support for alternative log aggregation mechanism

[ https://issues.apache.org/jira/browse/YARN-2082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007562#comment-14007562 ] Ming Ma commented on YARN-2082: --- Folks, thanks for the feedbacks and other jiras; quite

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870080#comment-13870080 ] Ming Ma commented on YARN-914: -- Junping/Luke, have you looked into the checkpointing framework

[jira] [Created] (YARN-1593) support out-of-proc AuxiliaryServices

Ming Ma created YARN-1593: - Summary: support out-of-proc AuxiliaryServices Key: YARN-1593 URL: https://issues.apache.org/jira/browse/YARN-1593 Project: Hadoop YARN Issue Type: Improvement

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Attachment: yarn container web page.png MR web page.png Ability to signal containers

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Attachment: (was: MR web page.png) Ability to signal containers

[jira] [Updated] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-445: - Attachment: (was: yarn container web page.png) Ability to signal containers

[jira] [Commented] (YARN-445) Ability to signal containers

[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902278#comment-13902278 ] Ming Ma commented on YARN-445: -- [Gera

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902291#comment-13902291 ] Ming Ma commented on YARN-221: -- [Chris

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909039#comment-13909039 ] Ming Ma commented on YARN-221: -- Jason, that is a good point. I wondered about the reason behind

[jira] [Created] (YARN-1750) TestNodeStatusUpdater#testNMRegistration incorrect

Ming Ma created YARN-1750: - Summary: TestNodeStatusUpdater#testNMRegistration incorrect Key: YARN-1750 URL: https://issues.apache.org/jira/browse/YARN-1750 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-1751) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

Ming Ma created YARN-1751: - Summary: Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing Key: YARN-1751 URL: https://issues.apache.org/jira/browse/YARN-1751 Project: Hadoop YARN

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909440#comment-13909440 ] Ming Ma commented on YARN-221: -- One thing to add we need the definition and policy on how to

[jira] [Created] (YARN-1756) capture the time when newApplication is called in RM

Ming Ma created YARN-1756: - Summary: capture the time when newApplication is called in RM Key: YARN-1756 URL: https://issues.apache.org/jira/browse/YARN-1756 Project: Hadoop YARN Issue Type:

[jira] [Moved] (YARN-2862) RM might not start if the machine was hard shutdown and FileSystemRMStateStore was used

[ https://issues.apache.org/jira/browse/YARN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma moved HADOOP-11305 to YARN-2862: Key: YARN-2862 (was: HADOOP-11305) Project: Hadoop YARN (was: Hadoop

[jira] [Commented] (YARN-2862) RM might not start if the machine was hard shutdown and FileSystemRMStateStore was used

[ https://issues.apache.org/jira/browse/YARN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212640#comment-14212640 ] Ming Ma commented on YARN-2862: --- Here are some possible ways to fix it. 1) Fix

[jira] [Commented] (YARN-2862) RM might not start if the machine was hard shutdown and FileSystemRMStateStore was used

[ https://issues.apache.org/jira/browse/YARN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214895#comment-14214895 ] Ming Ma commented on YARN-2862: --- Thanks, [~jira.shegalov], [~jianhe], [~zjshen]. I am able

[jira] [Created] (YARN-2899) Run TestDockerContainerExecutorWithMocks on Linux only

Ming Ma created YARN-2899: - Summary: Run TestDockerContainerExecutorWithMocks on Linux only Key: YARN-2899 URL: https://issues.apache.org/jira/browse/YARN-2899 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-2899) Run TestDockerContainerExecutorWithMocks on Linux only

[ https://issues.apache.org/jira/browse/YARN-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2899: -- Attachment: YARN-2899.patch Run TestDockerContainerExecutorWithMocks on Linux only

[jira] [Updated] (YARN-1996) Provide alternative policies for UNHEALTHY nodes.

[ https://issues.apache.org/jira/browse/YARN-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1996: -- Attachment: YARN-1996-2.patch [~jira.shegalov], [~maysamyabandeh] and I identified the root cause of

[jira] [Created] (YARN-2980) Move health check script related functionality to hadoop-common

Ming Ma created YARN-2980: - Summary: Move health check script related functionality to hadoop-common Key: YARN-2980 URL: https://issues.apache.org/jira/browse/YARN-2980 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254382#comment-14254382 ] Ming Ma commented on YARN-914: -- [~djp], thanks for working on this. It looks like we are going

[jira] [Commented] (YARN-2899) Run TestDockerContainerExecutorWithMocks on Linux only

[ https://issues.apache.org/jira/browse/YARN-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324495#comment-14324495 ] Ming Ma commented on YARN-2899: --- Thanks, Gera, Chris. Run

[jira] [Commented] (YARN-2980) Move health check script related functionality to hadoop-common

[ https://issues.apache.org/jira/browse/YARN-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294225#comment-14294225 ] Ming Ma commented on YARN-2980: --- [~varun_saxena], sorry for the late response. Couple

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266692#comment-14266692 ] Ming Ma commented on YARN-914: -- Thanks, Junping. The timeout is definitely necessary. * Sounds

[jira] [Commented] (YARN-2980) Move health check script related functionality to hadoop-common

[ https://issues.apache.org/jira/browse/YARN-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312660#comment-14312660 ] Ming Ma commented on YARN-2980: --- Thanks, Varun. LGTM. Move health check script related

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307724#comment-14307724 ] Ming Ma commented on YARN-914: -- I agree with Jason. It is easier if NM doesn't need to know

[jira] [Commented] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

[ https://issues.apache.org/jira/browse/YARN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368118#comment-14368118 ] Ming Ma commented on YARN-3212: --- bq. Do we want to consider DECOMMISSIONING nodes as not

[jira] [Commented] (YARN-3231) FairScheduler changing queueMaxRunningApps on the fly will cause all pending job stuck

[ https://issues.apache.org/jira/browse/YARN-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336697#comment-14336697 ] Ming Ma commented on YARN-3231: --- LGTM. FairScheduler changing queueMaxRunningApps on the

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-2.patch Thanks [~xgong]. Here is the rebased patch. Out of band heartbeats are sent only

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Labels: BB2015-05-RFC (was: ) Out of band heartbeats are sent only on container kill and possibly too early

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-trunk-v5.patch Here is the new patch with updated unit tests. NM should provide a way for AM

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551759#comment-14551759 ] Ming Ma commented on YARN-221: -- Thanks [~xgong]. You raise some valid points about abstraction.

[jira] [Commented] (YARN-2862) RM might not start if the machine was hard shutdown and FileSystemRMStateStore was used

[ https://issues.apache.org/jira/browse/YARN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596837#comment-14596837 ] Ming Ma commented on YARN-2862: --- Thanks, [~rohithsharma] and [~leftnoteasy]. Yes, YARN-3410

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601479#comment-14601479 ] Ming Ma commented on YARN-221: -- Here is the scenario. a) no applications want to over the

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601755#comment-14601755 ] Ming Ma commented on YARN-221: -- Thanks. [~vinodkv] and others, any additional suggestions for

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600546#comment-14600546 ] Ming Ma commented on YARN-221: -- Thanks Xuan! Regarding the default value for the policy, we

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598663#comment-14598663 ] Ming Ma commented on YARN-221: -- Thanks [~xgong]. How about the followings? * Allow

[jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId

[ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616120#comment-14616120 ] Ming Ma commented on YARN-3445: --- Thanks Junping. Can you please check if it really needs to

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708582#comment-14708582 ] Ming Ma commented on YARN-221: -- +1 on the addendum patch. NM should provide a way for AM to

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702607#comment-14702607 ] Ming Ma commented on YARN-221: -- Thanks Xuan. I have linked the newly created MR jira. NM

[jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId

[ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606766#comment-14606766 ] Ming Ma commented on YARN-3445: --- Thanks [~djp]. Quick questions: * Regarding the extra

  1   2   >