[jira] [Commented] (YARN-2960) Add documentation for the YARN shared cache

[ https://issues.apache.org/jira/browse/YARN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192216#comment-16192216 ] Ming Ma commented on YARN-2960: --- +1 > Add documentation for the YARN shared cache >

[jira] [Commented] (YARN-2960) Add documentation for the YARN shared cache

[ https://issues.apache.org/jira/browse/YARN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191441#comment-16191441 ] Ming Ma commented on YARN-2960: --- Thanks [~ctrezzo]. Looks good overview. For the configuratio

[jira] [Commented] (YARN-5464) Server-Side NM Graceful Decommissioning with RM HA

[ https://issues.apache.org/jira/browse/YARN-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157232#comment-16157232 ] Ming Ma commented on YARN-5464: --- I want to bring up the support for relative timeout value at

[jira] [Commented] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

[ https://issues.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157204#comment-16157204 ] Ming Ma commented on YARN-5536: --- Sorry for the delayed response. I have been busy and won't h

[jira] [Resolved] (YARN-1038) LocalizationProtocolPBClientImpl RPC failing

[ https://issues.apache.org/jira/browse/YARN-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma resolved YARN-1038. --- Resolution: Cannot Reproduce trunk branch no longer has this problem. [~tucu00] if you can repro with the late

[jira] [Updated] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

[ https://issues.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-5536: -- Priority: Critical (was: Major) Moved the priority to Critical based on the following discussion with [~djp]. Y

[jira] [Commented] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

[ https://issues-test.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090128#comment-16090128 ] Ming Ma commented on YARN-5536: --- I am not suggesting removing the previous format suppor

[jira] [Commented] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

[ https://issues-test.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090126#comment-16090126 ] Ming Ma commented on YARN-5536: --- Per discussion in YARN-4676, current timeout config sup

[jira] [Commented] (YARN-1038) LocalizationProtocolPBClientImpl RPC failing

[ https://issues-test.apache.org/jira/browse/YARN-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090124#comment-16090124 ] Ming Ma commented on YARN-1038: --- Given this blocker was opened many years ago and there

[jira] [Created] (YARN-6910) Increase RM audit log coverage

Ming Ma created YARN-6910: - Summary: Increase RM audit log coverage Key: YARN-6910 URL: https://issues.apache.org/jira/browse/YARN-6910 Project: Hadoop YARN Issue Type: Improvement Report

[jira] [Commented] (YARN-5396) YARN large file broadcast service

[ https://issues.apache.org/jira/browse/YARN-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024164#comment-16024164 ] Ming Ma commented on YARN-5396: --- Thanks [~aplusplus]! Is there any new progress on this? Inte

[jira] [Commented] (YARN-1197) Support changing resources of an allocated container

[ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001676#comment-16001676 ] Ming Ma commented on YARN-1197: --- Thanks for info [~tdbaker], [~jianhe], [~asuresh], [~kasha]!

[jira] [Commented] (YARN-1197) Support changing resources of an allocated container

[ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989682#comment-15989682 ] Ming Ma commented on YARN-1197: --- Thanks for the feature! Is the fair scheduler support availa

[jira] [Updated] (YARN-6004) Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines

[ https://issues.apache.org/jira/browse/YARN-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-6004: -- Fix Version/s: (was: 3.0.0-alpha3) (was: 2.9.0) > Refactor TestResourceLocalizationSer

[jira] [Reopened] (YARN-6004) Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines

[ https://issues.apache.org/jira/browse/YARN-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma reopened YARN-6004: --- > Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer > so that it is less than 150 lines

[jira] [Commented] (YARN-5536) Multiple format support (JSON, etc.) for exclude node file in NM graceful decommission with timeout

[ https://issues.apache.org/jira/browse/YARN-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703441#comment-15703441 ] Ming Ma commented on YARN-5536: --- I don't have any immediate plan to work on it yet. HDFS-9005

[jira] [Commented] (YARN-5464) Server-Side NM Graceful Decommissioning with RM HA

[ https://issues.apache.org/jira/browse/YARN-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472088#comment-15472088 ] Ming Ma commented on YARN-5464: --- Maybe this was discussed in the other jira, currently the ti

[jira] [Commented] (YARN-4676) Automatic and Asynchronous Decommissioning Nodes Status Tracking

[ https://issues.apache.org/jira/browse/YARN-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472063#comment-15472063 ] Ming Ma commented on YARN-4676: --- Thanks all! [~djp], sorry I missed your earlier question abo

[jira] [Commented] (YARN-4676) Automatic and Asynchronous Decommissioning Nodes Status Tracking

[ https://issues.apache.org/jira/browse/YARN-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415588#comment-15415588 ] Ming Ma commented on YARN-4676: --- Did we talking about moving timeout configuration piece out

[jira] [Commented] (YARN-1529) Add Localization overhead metrics to NM

[ https://issues.apache.org/jira/browse/YARN-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388826#comment-15388826 ] Ming Ma commented on YARN-1529: --- With ATS v2 in trunk and other frameworks such as Tez wantin

[jira] [Created] (YARN-5365) Add support for YARN Shared Cache

Ming Ma created YARN-5365: - Summary: Add support for YARN Shared Cache Key: YARN-5365 URL: https://issues.apache.org/jira/browse/YARN-5365 Project: Hadoop YARN Issue Type: Improvement Rep

[jira] [Commented] (YARN-867) Isolation of failures in aux services

[ https://issues.apache.org/jira/browse/YARN-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373658#comment-15373658 ] Ming Ma commented on YARN-867: -- Will this be simplified if we have YARN-1593? > Isolation of f

[jira] [Commented] (YARN-4676) Automatic and Asynchronous Decommissioning Nodes Status Tracking

[ https://issues.apache.org/jira/browse/YARN-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373342#comment-15373342 ] Ming Ma commented on YARN-4676: --- Sorry for joining the discussion late. For the timeout confi

[jira] [Updated] (YARN-5072) Support comma separated list of includes and excludes files

[ https://issues.apache.org/jira/browse/YARN-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-5072: -- Attachment: YARN-5072.patch Here is the draft path. YARN and HDFS use HostsFileReader differently. The patch inc

[jira] [Commented] (YARN-5072) Support comma separated list of includes and excludes files

[ https://issues.apache.org/jira/browse/YARN-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281157#comment-15281157 ] Ming Ma commented on YARN-5072: --- Thanks [~raviprak]. I have updated the description based on

[jira] [Updated] (YARN-5072) Support comma separated list of includes and excludes files

[ https://issues.apache.org/jira/browse/YARN-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-5072: -- Description: When a yarn cluster shares the same hosts as the underlying HDFS cluster, we have {{yarn.resourcema

[jira] [Created] (YARN-5072) Support comma separated list of includes and excludes files

Ming Ma created YARN-5072: - Summary: Support comma separated list of includes and excludes files Key: YARN-5072 URL: https://issues.apache.org/jira/browse/YARN-5072 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-4773) Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled

[ https://issues.apache.org/jira/browse/YARN-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185401#comment-15185401 ] Ming Ma commented on YARN-4773: --- [~jlowe], can you please confirm if it has been fixed by YAR

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167718#comment-15167718 ] Ming Ma commented on YARN-4720: --- Even though all tests passed by jenkins, but if you run the

[jira] [Commented] (YARN-4735) Remove stale LogAggregationReport from NM's context

[ https://issues.apache.org/jira/browse/YARN-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167456#comment-15167456 ] Ming Ma commented on YARN-4735: --- I just read the code again. {{NodeStatusUpdaterImpl}} 's {{

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167435#comment-15167435 ] Ming Ma commented on YARN-4720: --- +1 on the latest patch. I will wait until tomorrow to commit

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166691#comment-15166691 ] Ming Ma commented on YARN-4720: --- ah, that is a good point. So for long running service, the

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166575#comment-15166575 ] Ming Ma commented on YARN-4720: --- It seems that {{LogAggregationStatus.RUNNING}} implies the l

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163918#comment-15163918 ] Ming Ma commented on YARN-4720: --- Thanks [~hex108] for the update. The patch looks good overal

[jira] [Commented] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159663#comment-15159663 ] Ming Ma commented on YARN-2046: --- Thanks [~jlowe] and [~xgong]! > Out of band heartbeats are

[jira] [Commented] (YARN-4720) Skip unnecessary NN operations in log aggregation

[ https://issues.apache.org/jira/browse/YARN-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158999#comment-15158999 ] Ming Ma commented on YARN-4720: --- Thanks [~hex108] for the patch. It addresses the first scena

[jira] [Created] (YARN-4720) Skip unnecessary NN operations in log aggregation

Ming Ma created YARN-4720: - Summary: Skip unnecessary NN operations in log aggregation Key: YARN-4720 URL: https://issues.apache.org/jira/browse/YARN-4720 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-4690) Skip object allocation in FSAppAttempt#getResourceUsage when possible

[ https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157397#comment-15157397 ] Ming Ma commented on YARN-4690: --- Thanks [~sjlee0] and [~kasha]! > Skip object allocation in

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-branch-2.6.patch YARN-2046-branch-2.7.patch Thanks [~jlowe]. Agree it is us

[jira] [Commented] (YARN-4690) Skip object allocation in FSAppAttempt#getResourceUsage when possible

[ https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145321#comment-15145321 ] Ming Ma commented on YARN-4690: --- The failed tests pass locally and aren't related. > Skip ob

[jira] [Updated] (YARN-4690) Skip object allocation in FSAppAttempt#getResourceUsage when possible

[ https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-4690: -- Attachment: YARN-4690.patch Here is the draft patch. After we fix YARN-4691, this jira becomes less important. B

[jira] [Created] (YARN-4691) Cache resource usage at FSLeafQueue level

Ming Ma created YARN-4691: - Summary: Cache resource usage at FSLeafQueue level Key: YARN-4691 URL: https://issues.apache.org/jira/browse/YARN-4691 Project: Hadoop YARN Issue Type: Improvement

[jira] [Created] (YARN-4690) Skip object allocation in FSAppAttempt#getResourceUsage when possible

Ming Ma created YARN-4690: - Summary: Skip object allocation in FSAppAttempt#getResourceUsage when possible Key: YARN-4690 URL: https://issues.apache.org/jira/browse/YARN-4690 Project: Hadoop YARN Is

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-5.patch Thanks [~jlowe]! Here is the updated patch with your suggestion. > Out of band hea

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-4.patch The TestLogAggregationService failure is unrelated. It passes locally. The update

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-3.patch With out-of-band heartbeat we can afford to set larger NM -> RM heartbeat interval

[jira] [Commented] (YARN-4612) Fix rumen and scheduler load simulator handle killed tasks properly

[ https://issues.apache.org/jira/browse/YARN-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118640#comment-15118640 ] Ming Ma commented on YARN-4612: --- Thanks [~xgong]. > Fix rumen and scheduler load simulator h

[jira] [Resolved] (YARN-4620) getApplicationAttemptReport could throw exception in the case of unmanaged app

[ https://issues.apache.org/jira/browse/YARN-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma resolved YARN-4620. --- Resolution: Duplicate Thanks [~rohithsharma]! Yes it is dup. I have verified the patch in YARN-4411 has fixed

[jira] [Created] (YARN-4620) getApplicationAttemptReport could throw exception in the case of unmanaged app

Ming Ma created YARN-4620: - Summary: getApplicationAttemptReport could throw exception in the case of unmanaged app Key: YARN-4620 URL: https://issues.apache.org/jira/browse/YARN-4620 Project: Hadoop YARN

[jira] [Created] (YARN-4619) Add support to scheduler load simulator to run NM and AM simulation separately from the RM process

Ming Ma created YARN-4619: - Summary: Add support to scheduler load simulator to run NM and AM simulation separately from the RM process Key: YARN-4619 URL: https://issues.apache.org/jira/browse/YARN-4619 Proj

[jira] [Commented] (YARN-4611) Fix scheduler load simulator to support multi-layer network location

[ https://issues.apache.org/jira/browse/YARN-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109754#comment-15109754 ] Ming Ma commented on YARN-4611: --- Thanks [~xgong]! > Fix scheduler load simulator to support

[jira] [Updated] (YARN-4612) Fix rumen and scheduler load simulator handle killed tasks properly

[ https://issues.apache.org/jira/browse/YARN-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-4612: -- Attachment: YARN-4612-2.patch The json had a new job with zero task attempt which is the new unit test. The upda

[jira] [Updated] (YARN-4611) Fix scheduler load simulator to support multi-layer network location

[ https://issues.apache.org/jira/browse/YARN-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-4611: -- Attachment: YARN-4611-2.patch New patch to fix TestNMSimulator. > Fix scheduler load simulator to support multi-

[jira] [Updated] (YARN-4612) Fix rumen and scheduler load simulator handle killed tasks properly

[ https://issues.apache.org/jira/browse/YARN-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-4612: -- Attachment: YARN-4612.patch Here is the draft patch. Also tested it with actual data. > Fix rumen and scheduler

[jira] [Created] (YARN-4612) Fix rumen and scheduler load simulator handle killed tasks properly

Ming Ma created YARN-4612: - Summary: Fix rumen and scheduler load simulator handle killed tasks properly Key: YARN-4612 URL: https://issues.apache.org/jira/browse/YARN-4612 Project: Hadoop YARN Issu

[jira] [Updated] (YARN-4611) Fix scheduler load simulator to support multi-layer network location

[ https://issues.apache.org/jira/browse/YARN-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-4611: -- Attachment: YARN-4611.patch Here is the draft patch. Also tested it with actual rumen trace. > Fix scheduler loa

[jira] [Created] (YARN-4611) Fix scheduler load simulator to support multi-layer network location

Ming Ma created YARN-4611: - Summary: Fix scheduler load simulator to support multi-layer network location Key: YARN-4611 URL: https://issues.apache.org/jira/browse/YARN-4611 Project: Hadoop YARN Iss

[jira] [Commented] (YARN-4024) YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat

[ https://issues.apache.org/jira/browse/YARN-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075619#comment-15075619 ] Ming Ma commented on YARN-4024: --- Thanks for the good improvement [~leftnoteasy], [~zhiguohong

[jira] [Commented] (YARN-4422) Generic AHS sometimes doesn't show started, node, or logs on App page

[ https://issues.apache.org/jira/browse/YARN-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051548#comment-15051548 ] Ming Ma commented on YARN-4422: --- Thanks [~eepayne]. > Generic AHS sometimes doesn't show sta

[jira] [Commented] (YARN-4422) Generic AHS sometimes doesn't show started, node, or logs on App page

[ https://issues.apache.org/jira/browse/YARN-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15049740#comment-15049740 ] Ming Ma commented on YARN-4422: --- Thanks! Will this fix address MAPREDUCE-5502 or MAPREDUCE-44

[jira] [Updated] (YARN-2913) Fair scheduler should have ability to set MaxResourceDefault for each queue

[ https://issues.apache.org/jira/browse/YARN-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2913: -- Issue Type: Improvement (was: Bug) > Fair scheduler should have ability to set MaxResourceDefault for each queue

[jira] [Commented] (YARN-2913) Fair scheduler should have ability to set MaxResourceDefault for each queue

[ https://issues.apache.org/jira/browse/YARN-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970092#comment-14970092 ] Ming Ma commented on YARN-2913: --- Thanks [~l201514]. Can you please update FairScheduler.md?

[jira] [Commented] (YARN-2913) Fair scheduler should have ability to set MaxResourceDefault for each queue

[ https://issues.apache.org/jira/browse/YARN-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14967901#comment-14967901 ] Ming Ma commented on YARN-2913: --- The patch looks good. Nit: maybe you want to rename {{queueM

[jira] [Commented] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942053#comment-14942053 ] Ming Ma commented on YARN-1897: --- Thanks [~xgong] for helping with design, code review and the

[jira] [Updated] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Attachment: YARN-1897-8.patch Thanks [~xgong]. Here is the rebase. The failed unit tests aren't related. > CLI a

[jira] [Commented] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906625#comment-14906625 ] Ming Ma commented on YARN-1897: --- Thanks [~xgong]. Regarding the diagnosis, do you want to al

[jira] [Updated] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Attachment: YARN-1897-7.patch Thanks [~djp]! bq. Number of preempted containers won't be count as container fai

[jira] [Updated] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Attachment: YARN-1897-6.patch Thanks [~djp]! Yes, the approach taken in YARN-4131 is simpler by leveraging the e

[jira] [Updated] (YARN-1897) CLI and core support for signal container functionality

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Summary: CLI and core support for signal container functionality (was: Define SignalContainerRequest and SignalC

[jira] [Updated] (YARN-1897) Define SignalContainerRequest and SignalContainerResponse

[ https://issues.apache.org/jira/browse/YARN-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-1897: -- Attachment: YARN-1897-5.patch Based on the offline discussion with [~djp] [~ste...@apache.org] [~xgong] w.r.t. t

[jira] [Commented] (YARN-4131) Add API and CLI to kill container on given containerId

[ https://issues.apache.org/jira/browse/YARN-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737912#comment-14737912 ] Ming Ma commented on YARN-4131: --- [~djp], The discussion of YARN-445 has involved over time. T

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708582#comment-14708582 ] Ming Ma commented on YARN-221: -- +1 on the addendum patch. > NM should provide a way for AM to

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702607#comment-14702607 ] Ming Ma commented on YARN-221: -- Thanks Xuan. I have linked the newly created MR jira. > NM sho

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697314#comment-14697314 ] Ming Ma commented on YARN-221: -- The unit test failures aren't related. The tests pass on the lo

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-9.patch I had offline discussion with Xuan about the API. To support this as public interface

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661140#comment-14661140 ] Ming Ma commented on YARN-221: -- My main motivation of reusing ContainerTerminationContext is to

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659286#comment-14659286 ] Ming Ma commented on YARN-221: -- That sounds a good idea. How about using the existing Containe

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-8.patch The javac warning isn't related to this patch. That is due to {{TestAuxServices}} cast

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-7.patch Thanks [~xgong]! Here is the updated patch with your suggestions. {{ContainerLogAggreg

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-6.patch [~xgong] and others, here is the draft patch based on the new design. Besides the abov

[jira] [Commented] (YARN-3936) Add metrics for RMStateStore

[ https://issues.apache.org/jira/browse/YARN-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632186#comment-14632186 ] Ming Ma commented on YARN-3936: --- Thanks [~sunilg]! Please go ahead. > Add metrics for RMStat

[jira] [Commented] (YARN-3934) Application with large ApplicationSubmissionContext can cause RM to exit when ZK store is used

[ https://issues.apache.org/jira/browse/YARN-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632017#comment-14632017 ] Ming Ma commented on YARN-3934: --- This is due to a single ASC object size. You can repro this

[jira] [Created] (YARN-3936) Add metrics for RMStateStore

Ming Ma created YARN-3936: - Summary: Add metrics for RMStateStore Key: YARN-3936 URL: https://issues.apache.org/jira/browse/YARN-3936 Project: Hadoop YARN Issue Type: Improvement Reporter

[jira] [Created] (YARN-3935) Support compression for RM HA ApplicationStateData

Ming Ma created YARN-3935: - Summary: Support compression for RM HA ApplicationStateData Key: YARN-3935 URL: https://issues.apache.org/jira/browse/YARN-3935 Project: Hadoop YARN Issue Type: Improvemen

[jira] [Created] (YARN-3934) Application with large ApplicationSubmissionContext can cause RM to exit when ZK store is used

Ming Ma created YARN-3934: - Summary: Application with large ApplicationSubmissionContext can cause RM to exit when ZK store is used Key: YARN-3934 URL: https://issues.apache.org/jira/browse/YARN-3934 Project:

[jira] [Commented] (YARN-2578) NM does not failover timely if RM node network connection fails

[ https://issues.apache.org/jira/browse/YARN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629940#comment-14629940 ] Ming Ma commented on YARN-2578: --- Thanks [~iwasakims]. Is it similar to HADOOP-11252? Given yo

[jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId

[ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620855#comment-14620855 ] Ming Ma commented on YARN-3445: --- +1 on the latest patch. Thanks Junping. I will wait until to

[jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId

[ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616120#comment-14616120 ] Ming Ma commented on YARN-3445: --- Thanks Junping. Can you please check if it really needs to t

[jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId

[ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606766#comment-14606766 ] Ming Ma commented on YARN-3445: --- Thanks [~djp]. Quick questions: * Regarding the extra memor

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601755#comment-14601755 ] Ming Ma commented on YARN-221: -- Thanks. [~vinodkv] and others, any additional suggestions for t

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601479#comment-14601479 ] Ming Ma commented on YARN-221: -- Here is the scenario. a) no applications want to over the defau

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600546#comment-14600546 ] Ming Ma commented on YARN-221: -- Thanks Xuan! Regarding the default value for the policy, we wan

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598663#comment-14598663 ] Ming Ma commented on YARN-221: -- Thanks [~xgong]. How about the followings? * Allow application

[jira] [Commented] (YARN-2862) RM might not start if the machine was hard shutdown and FileSystemRMStateStore was used

[ https://issues.apache.org/jira/browse/YARN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596837#comment-14596837 ] Ming Ma commented on YARN-2862: --- Thanks, [~rohithsharma] and [~leftnoteasy]. Yes, YARN-3410 w

[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551759#comment-14551759 ] Ming Ma commented on YARN-221: -- Thanks [~xgong]. You raise some valid points about abstraction.

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-trunk-v5.patch Here is the new patch with updated unit tests. > NM should provide a way for AM

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-trunk-v4.patch Updated patch to fix warnings. > NM should provide a way for AM to tell it not

[jira] [Updated] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.

[ https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-221: - Attachment: YARN-221-trunk-v3.patch Thanks [~gtCarrera9]. Here is the rebased patch. > NM should provide a way for

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Labels: BB2015-05-RFC (was: ) > Out of band heartbeats are sent only on container kill and possibly too early >

[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early

[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-2.patch Thanks [~xgong]. Here is the rebased patch. > Out of band heartbeats are sent only

  1   2   >