[jira] [Commented] (YARN-3064) TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout in trunk

[ https://issues.apache.org/jira/browse/YARN-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279933#comment-14279933 ] Junping Du commented on YARN-3064: -- v2 patch LGTM. +1. Will commit it shortly.

[jira] [Updated] (YARN-3064) TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout

[ https://issues.apache.org/jira/browse/YARN-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3064: - Summary: TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout

[jira] [Updated] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command

[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3070: - Hadoop Flags: Reviewed Thanks [~hitliuyi] for review. Committing v2 patch in. TestRMAdminCLI#testHelp

[jira] [Assigned] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command

[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned YARN-3070: Assignee: Junping Du TestRMAdminCLI#testHelp fails for transitionToActive command

[jira] [Updated] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command

[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3070: - Attachment: YARN-3070.patch Thanks [~te...@apache.org] for reporting this. Deliver a quick patch to fix

[jira] [Commented] (YARN-3064) TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout in trunk

[ https://issues.apache.org/jira/browse/YARN-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278436#comment-14278436 ] Junping Du commented on YARN-3064: -- Patch looks good to me. However, for failures in

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288644#comment-14288644 ] Junping Du commented on YARN-914: - Sorry for replying late. These are all good points, a

[jira] [Updated] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command

[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3070: - Attachment: YARN-3070-v2.patch Fix a typo of String.format() which cause the test failure.

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316606#comment-14316606 ] Junping Du commented on YARN-914: - bq. I do agree with Vinod that there should minimally be

[jira] [Updated] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-914: Attachment: Gracefully Decommission of NodeManager (v2).pdf Update proposal to reflect what we discussed

[jira] [Commented] (YARN-2079) Recover NonAggregatingLogHandler state upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317233#comment-14317233 ] Junping Du commented on YARN-2079: -- Hi [~jlowe], sorry for missing this. I will review it

[jira] [Commented] (YARN-3160) Non-atomic operation on nodeUpdateQueue in RMNodeImpl

[ https://issues.apache.org/jira/browse/YARN-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314482#comment-14314482 ] Junping Du commented on YARN-3160: -- Didn't see these failures in testReport. Kick off

[jira] [Updated] (YARN-3194) After NM restart,completed containers are not released by RM which are sent during NM registration

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3194: - Affects Version/s: (was: 2.6.0) 2.7.0 Update affect version to be 2.7. May be a

[jira] [Commented] (YARN-3194) After NM restart,completed containers are not released by RM which are sent during NM registration

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325212#comment-14325212 ] Junping Du commented on YARN-3194: -- bq. I didn't see this problem originally, but I

[jira] [Commented] (YARN-3194) After NM restart,completed containers are not released by RM which are sent during NM registration

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325264#comment-14325264 ] Junping Du commented on YARN-3194: -- Should be a blocker to 2.7 as it blocks rolling

[jira] [Updated] (YARN-3194) After NM restart,completed containers are not released by RM which are sent during NM registration

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3194: - Priority: Blocker (was: Major) After NM restart,completed containers are not released by RM which are

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323124#comment-14323124 ] Junping Du commented on YARN-914: - Thanks [~jlowe] for review and comments! bq. Nit: How

[jira] [Updated] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

[ https://issues.apache.org/jira/browse/YARN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3212: - Attachment: RMNodeImpl - new.png Attache the new state transition diagram for RMNode. RMNode State

[jira] [Created] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

Junping Du created YARN-3212: Summary: RMNode State Transition Update with DECOMMISSIONING state Key: YARN-3212 URL: https://issues.apache.org/jira/browse/YARN-3212 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-2079) Recover NonAggregatingLogHandler state upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318741#comment-14318741 ] Junping Du commented on YARN-2079: -- Thanks [~jlowe] for addressing my comments in 003

[jira] [Commented] (YARN-3188) yarn application --list should list all the applications ( Not only submitted,accepted and running)

[ https://issues.apache.org/jira/browse/YARN-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318033#comment-14318033 ] Junping Du commented on YARN-3188: -- I agree with Rohit. I think we do this intentionally

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324390#comment-14324390 ] Junping Du commented on YARN-914: - bq. The main point I'm trying to make here is that we

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327610#comment-14327610 ] Junping Du commented on YARN-914: - Break down this feature into sub-JIRAs. Support

[jira] [Updated] (YARN-914) (Umbrella) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-914: Summary: (Umbrella) Support graceful decommission of nodemanager (was: Support graceful decommission of

[jira] [Created] (YARN-3226) UI changes for decommissioning node

Junping Du created YARN-3226: Summary: UI changes for decommissioning node Key: YARN-3226 URL: https://issues.apache.org/jira/browse/YARN-3226 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Updated] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-914: Attachment: GracefullyDecommissionofNodeManagerv3.pdf Update proposal to incorporate most comments above,

[jira] [Created] (YARN-3224) Notify AM with containers (on decommissioning node) could be preempted after timeout.

Junping Du created YARN-3224: Summary: Notify AM with containers (on decommissioning node) could be preempted after timeout. Key: YARN-3224 URL: https://issues.apache.org/jira/browse/YARN-3224 Project:

[jira] [Created] (YARN-3223) Resource update during NM graceful decommission

Junping Du created YARN-3223: Summary: Resource update during NM graceful decommission Key: YARN-3223 URL: https://issues.apache.org/jira/browse/YARN-3223 Project: Hadoop YARN Issue Type:

[jira] [Created] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI

Junping Du created YARN-3225: Summary: New parameter or CLI for decommissioning node gracefully in RMAdmin CLI Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN

[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI

[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327807#comment-14327807 ] Junping Du commented on YARN-3225: -- Thanks [~sunilg] for the comments! Yes. I mean mradmin

[jira] [Commented] (YARN-3224) Notify AM with containers (on decommissioning node) could be preempted after timeout.

[ https://issues.apache.org/jira/browse/YARN-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327811#comment-14327811 ] Junping Du commented on YARN-3224: -- Sure. Please go ahead to take on this JIRA. Thanks

[jira] [Commented] (YARN-3194) After NM restart,completed containers are not released which are sent during NM registration

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323980#comment-14323980 ] Junping Du commented on YARN-3194: -- I think NM after restarted will try to relaunch these

[jira] [Commented] (YARN-41) The RM should handle the graceful shutdown of the NM.

[ https://issues.apache.org/jira/browse/YARN-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296222#comment-14296222 ] Junping Du commented on YARN-41: [~devaraj.k], thanks for updating the patch! [~vinodkv] is

[jira] [Resolved] (YARN-2680) Node shouldn't be listed as RUNNING when NM daemon is stop even when recovery work is disabled.

[ https://issues.apache.org/jira/browse/YARN-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved YARN-2680. -- Resolution: Duplicate Node shouldn't be listed as RUNNING when NM daemon is stop even when recovery

[jira] [Updated] (YARN-2680) Node shouldn't be listed as RUNNING when NM daemon is stop even when recovery work is disabled.

[ https://issues.apache.org/jira/browse/YARN-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2680: - Summary: Node shouldn't be listed as RUNNING when NM daemon is stop even when recovery work is disabled.

[jira] [Commented] (YARN-2680) Node shouldn't be listed as RUNNING when NM daemon is stop even when recovery work is enabled.

[ https://issues.apache.org/jira/browse/YARN-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296209#comment-14296209 ] Junping Du commented on YARN-2680: -- Hi [~jlowe], I think I meant that Node shouldn't be

[jira] [Updated] (YARN-2571) RM to support YARN registry

[ https://issues.apache.org/jira/browse/YARN-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2571: - Target Version/s: 2.7.0 (was: 2.6.0) RM to support YARN registry

[jira] [Commented] (YARN-3019) Enable RM work-preserving restart by default

[ https://issues.apache.org/jira/browse/YARN-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272792#comment-14272792 ] Junping Du commented on YARN-3019: -- bq. To clarify: this jira is to flip recovery mode to

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-313: Attachment: YARN-313-v3.patch Resync the patch to latest trunk. Add Admin API for supporting node resource

[jira] [Commented] (YARN-3019) Enable RM work-preserving restart by default

[ https://issues.apache.org/jira/browse/YARN-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275278#comment-14275278 ] Junping Du commented on YARN-3019: -- bq. The final goal is to support work-preserving

[jira] [Commented] (YARN-3159) DOCKER_IMAGE_PATTERN should support multilayered path of docker images

[ https://issues.apache.org/jira/browse/YARN-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312354#comment-14312354 ] Junping Du commented on YARN-3159: -- Thanks [~guoleitao]. Please add one layer case also

[jira] [Commented] (YARN-3160) Non-atomic operation on nodeUpdateQueue in RMNodeImpl

[ https://issues.apache.org/jira/browse/YARN-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312342#comment-14312342 ] Junping Du commented on YARN-3160: -- Hi [~chengbing.liu], thanks for reporting the issue

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312525#comment-14312525 ] Junping Du commented on YARN-914: - Thanks for review and comments, [~xgong], [~jlowe] and

[jira] [Commented] (YARN-2799) cleanup TestLogAggregationService based on the change in YARN-90

[ https://issues.apache.org/jira/browse/YARN-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320096#comment-14320096 ] Junping Du commented on YARN-2799: -- Patch looks good in overall. However, a comment for a

[jira] [Commented] (YARN-2749) Some testcases from TestLogAggregationService fails in trunk

[ https://issues.apache.org/jira/browse/YARN-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14320067#comment-14320067 ] Junping Du commented on YARN-2749: -- The patch looks reasonable. +1. Will commit it

[jira] [Commented] (YARN-914) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314653#comment-14314653 ] Junping Du commented on YARN-914: - Thanks [~vinodkv] for comments! bq. IAC, I think we

[jira] [Commented] (YARN-3173) start-yarn.sh script can't aware how many RMs to be started.

[ https://issues.apache.org/jira/browse/YARN-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316099#comment-14316099 ] Junping Du commented on YARN-3173: -- Setup RM HA also involve other manual steps, e.g.

[jira] [Commented] (YARN-3160) Non-atomic operation on nodeUpdateQueue in RMNodeImpl

[ https://issues.apache.org/jira/browse/YARN-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316068#comment-14316068 ] Junping Du commented on YARN-3160: -- +1. Patch looks good to me. Committing it now.

[jira] [Commented] (YARN-1580) Documentation error regarding container-allocation.expiry-interval-ms

[ https://issues.apache.org/jira/browse/YARN-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316091#comment-14316091 ] Junping Du commented on YARN-1580: -- +1. Patch looks good to me. Will commit it shortly.

[jira] [Updated] (YARN-3159) DOCKER_IMAGE_PATTERN should support multilayered path of docker images

[ https://issues.apache.org/jira/browse/YARN-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3159: - Assignee: Leitao Guo DOCKER_IMAGE_PATTERN should support multilayered path of docker images

[jira] [Commented] (YARN-3159) DOCKER_IMAGE_PATTERN should support multilayered path of docker images

[ https://issues.apache.org/jira/browse/YARN-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312149#comment-14312149 ] Junping Du commented on YARN-3159: -- Hi [~guoleitao], thanks for delivering a patch here.

[jira] [Commented] (YARN-41) The RM should handle the graceful shutdown of the NM.

[ https://issues.apache.org/jira/browse/YARN-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312225#comment-14312225 ] Junping Du commented on YARN-41: I think I could have a little misunderstand before. After

[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers

[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318576#comment-14318576 ] Junping Du commented on YARN-3031: -- Thanks [~vrushalic] for updating the patch and

[jira] [Commented] (YARN-2079) Recover NonAggregatingLogHandler state upon nodemanager restart

[ https://issues.apache.org/jira/browse/YARN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318461#comment-14318461 ] Junping Du commented on YARN-2079: -- Thanks [~jlowe] for updating the patch! The patch

[jira] [Commented] (YARN-2994) Document work-preserving RM restart

[ https://issues.apache.org/jira/browse/YARN-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318540#comment-14318540 ] Junping Du commented on YARN-2994: -- Thanks [~jianhe] for delivering a documentation patch

[jira] [Commented] (YARN-2799) cleanup TestLogAggregationService based on the change in YARN-90

[ https://issues.apache.org/jira/browse/YARN-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14322656#comment-14322656 ] Junping Du commented on YARN-2799: -- Latest patch looks good to me. Kick off Jenkins test

[jira] [Commented] (YARN-3033) [Aggregator wireup] Implement NM starting the ATS writer companion

[ https://issues.apache.org/jira/browse/YARN-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328365#comment-14328365 ] Junping Du commented on YARN-3033: -- Thanks [~gtCarrera9] for delivering a proposal here

[jira] [Commented] (YARN-3194) After NM restart, RM should handle NMCotainerStatuses sent by NM while registering if NM is Reconnected node

[ https://issues.apache.org/jira/browse/YARN-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328333#comment-14328333 ] Junping Du commented on YARN-3194: -- lgtm three. :-) After NM restart, RM should handle

[jira] [Commented] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366950#comment-14366950 ] Junping Du commented on YARN-3039: -- Thanks [~zjshen] and [~sjlee0] for review!

[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367031#comment-14367031 ] Junping Du commented on YARN-3034: -- bq. As a further clarification, my problem is mainly

[jira] [Commented] (YARN-914) (Umbrella) Support graceful decommission of nodemanager

[ https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367873#comment-14367873 ] Junping Du commented on YARN-914: - Hi, can someone in watch list help to review patch in sub

[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level

[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363267#comment-14363267 ] Junping Du commented on YARN-3350: -- Thanks [~wilfreds] for reporting the issue and

[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level

[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363269#comment-14363269 ] Junping Du commented on YARN-3350: -- Sorry for typo: is LOG is not enabling debug level =

[jira] [Updated] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

[ https://issues.apache.org/jira/browse/YARN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3212: - Attachment: YARN-3212-v2.patch Fix two test failures in v2 version. The findbugs warnings are not related

[jira] [Commented] (YARN-3334) [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2.

[ https://issues.apache.org/jira/browse/YARN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363487#comment-14363487 ] Junping Du commented on YARN-3334: -- Hi [~Naganarasimha], Thanks for comments here, and

[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level

[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363456#comment-14363456 ] Junping Du commented on YARN-3350: -- Just get confirmation from Sandy at SPARK-5393.

[jira] [Updated] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3039: - Attachment: YARN-3039-v8.patch Incorporate [~zjshen]'s comments in v8 patch. For TestRPC, lets keep it

[jira] [Updated] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3039: - Attachment: YARN-3039-v7.patch Sync up offline with [~zjshen], with a few updates in v7 patch, comparing

[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI

[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370370#comment-14370370 ] Junping Du commented on YARN-3225: -- bq. I feel timeout would be enough, anyway we can wait

[jira] [Commented] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

[ https://issues.apache.org/jira/browse/YARN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370446#comment-14370446 ] Junping Du commented on YARN-3212: -- Thanks [~jlowe] and [~mingma] for review and comments!

[jira] [Commented] (YARN-3334) [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2.

[ https://issues.apache.org/jira/browse/YARN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372421#comment-14372421 ] Junping Du commented on YARN-3334: -- Hi [~gtCarrera9], thanks for the questions here. I

[jira] [Commented] (YARN-3209) RM and NM state should be added to the list of Hadoop Compatibility File list

[ https://issues.apache.org/jira/browse/YARN-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372900#comment-14372900 ] Junping Du commented on YARN-3209: -- And adding state of ShuffleHandler too. RM and NM

[jira] [Assigned] (YARN-3209) RM and NM state should be added to the list of Hadoop Compatibility File list

[ https://issues.apache.org/jira/browse/YARN-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned YARN-3209: Assignee: Junping Du RM and NM state should be added to the list of Hadoop Compatibility File list

[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI

[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369690#comment-14369690 ] Junping Du commented on YARN-3225: -- bq. I would prefer to have the name in sync with the

[jira] [Commented] (YARN-3269) Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path

[ https://issues.apache.org/jira/browse/YARN-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369729#comment-14369729 ] Junping Du commented on YARN-3269: -- Thanks [~xgong] for the patch, I will review your

[jira] [Commented] (YARN-3333) rename TimelineAggregator etc. to TimelineCollector

[ https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369865#comment-14369865 ] Junping Du commented on YARN-: -- Thanks [~sjlee0] for the patch! I have commit the

[jira] [Commented] (YARN-3033) [Aggregator wireup] Implement NM starting the standalone ATS writer companion

[ https://issues.apache.org/jira/browse/YARN-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369458#comment-14369458 ] Junping Du commented on YARN-3033: -- Also, RM may not start a aggregatorCollection (with

[jira] [Updated] (YARN-3334) [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2.

[ https://issues.apache.org/jira/browse/YARN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3334: - Attachment: YARN-3334-demo.patch Update a demo patch for putting some metrics info to new TimelineService.

[jira] [Commented] (YARN-3034) [Collector wireup] Implement RM starting its timeline collector

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369872#comment-14369872 ] Junping Du commented on YARN-3034: -- I have commit YARN- in. [~Naganarasimha], would

[jira] [Resolved] (YARN-3333) rename TimelineAggregator etc. to TimelineCollector

[ https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved YARN-. -- Resolution: Fixed Hadoop Flags: Reviewed rename TimelineAggregator etc. to TimelineCollector

[jira] [Commented] (YARN-3269) Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path

[ https://issues.apache.org/jira/browse/YARN-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371650#comment-14371650 ] Junping Du commented on YARN-3269: -- +1. v2 patch LGTM. Will commit it shortly.

[jira] [Commented] (YARN-3269) Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path

[ https://issues.apache.org/jira/browse/YARN-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371652#comment-14371652 ] Junping Du commented on YARN-3269: -- Kick off Jenkins test again in case any possible

[jira] [Updated] (YARN-3334) [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2.

[ https://issues.apache.org/jira/browse/YARN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3334: - Attachment: YARN-3334-v1.patch Upload v1 patch with adding test of TestDistributedShell to verify NM

[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level

[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363713#comment-14363713 ] Junping Du commented on YARN-3350: -- Thanks [~wilfreds] for updating the patch. +1 on

[jira] [Commented] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363706#comment-14363706 ] Junping Du commented on YARN-3039: -- Thanks [~zjshen] for clarification here! bq. It

[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363742#comment-14363742 ] Junping Du commented on YARN-3034: -- Thanks [~Naganarasimha] for updating the patch and

[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363747#comment-14363747 ] Junping Du commented on YARN-3034: -- bq. For the DistributedShell from Junping Du's

[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363762#comment-14363762 ] Junping Du commented on YARN-3034: -- bq. I confirm this as my comments above. I mean we

[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer

[ https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363785#comment-14363785 ] Junping Du commented on YARN-3034: -- OK. That sounds good. [Aggregator wireup] Implement

[jira] [Commented] (YARN-3212) RMNode State Transition Update with DECOMMISSIONING state

[ https://issues.apache.org/jira/browse/YARN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363793#comment-14363793 ] Junping Du commented on YARN-3212: -- The findbugs warning is not related to this patch and

[jira] [Commented] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365437#comment-14365437 ] Junping Du commented on YARN-3039: -- Thanks [~sjlee0], [~zjshen] and [~jianhe] for review

[jira] [Updated] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3039: - Attachment: YARN-3039-v6.patch Incorporate comments in v6 patch, verify end-to-end test work for

[jira] [Created] (YARN-3359) Recover aggregator (collector) list in RM failed over

Junping Du created YARN-3359: Summary: Recover aggregator (collector) list in RM failed over Key: YARN-3359 URL: https://issues.apache.org/jira/browse/YARN-3359 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI

[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352944#comment-14352944 ] Junping Du commented on YARN-3225: -- Thanks [~devaraj.k] for delivering the patch which is

[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters

[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352962#comment-14352962 ] Junping Du commented on YARN-3304: -- Agree that negative value sounds very odd. However, if

[jira] [Created] (YARN-3334) [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2.

Junping Du created YARN-3334: Summary: [Event Producers] NM start to posting some app related metrics in early POC stage of phase 2. Key: YARN-3334 URL: https://issues.apache.org/jira/browse/YARN-3334

[jira] [Commented] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357111#comment-14357111 ] Junping Du commented on YARN-3039: -- Thanks [~sjlee0]! Providing an end-to-end flow below

[jira] [Resolved] (YARN-3035) [Storage implementation] Create a test-only backing storage implementation for ATS writes

[ https://issues.apache.org/jira/browse/YARN-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved YARN-3035. -- Resolution: Duplicate Close it as duplicated as we already have a local file based storage

[jira] [Updated] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3039: - Attachment: YARN-3039-v3-core-changes-only.patch Attach the new v2 proposal to reflect what we discuss

[jira] [Updated] (YARN-3039) [Aggregator wireup] Implement ATS app-appgregator service discovery

[ https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3039: - Attachment: Service Discovery For Application Aggregator of ATS (v2).pdf [Aggregator wireup] Implement

<    3   4   5   6   7   8   9   10   11   12   >