[jira] [Assigned] (YARN-11489) Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker

[ https://issues.apache.org/jira/browse/YARN-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen reassigned YARN-11489: Assignee: Chun Chen > Fix memory leak of DelegationTokenRenewer futures in >

[jira] [Created] (YARN-11489) Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker

Chun Chen created YARN-11489: Summary: Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker Key: YARN-11489 URL: https://issues.apache.org/jira/browse/YARN-11489

[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433277#comment-16433277 ] Chun Chen commented on YARN-2674: - [~shaneku...@gmail.com] Please catch on the work here. I no longer work

[jira] [Commented] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589179#comment-14589179 ] Chun Chen commented on YARN-1983: - [~sidharta-s], thanks for letting me know the progress

[jira] [Commented] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587370#comment-14587370 ] Chun Chen commented on YARN-1983: - [~vinodkv], according to your suggestion, I propose the

[jira] [Commented] (YARN-3469) ZKRMStateStore: Avoid setting watches that are not required

[ https://issues.apache.org/jira/browse/YARN-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585394#comment-14585394 ] Chun Chen commented on YARN-3469: - Just a note. I saw ZOOKEEPER-706 has been resolved

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Attachment: YARN-2674.4.patch Distributed shell AM may re-launch containers if RM work preserving restart

[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574106#comment-14574106 ] Chun Chen commented on YARN-2674: - Upload a patch to fix test failures. Distributed shell

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Attachment: YARN-2674.5.patch Upload YARN-2674.5.patch to remove unnecessary synchronize. Distributed

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Attachment: YARN-2674.3.patch Distributed shell AM may re-launch containers if RM work preserving restart

[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572535#comment-14572535 ] Chun Chen commented on YARN-2674: - Upload YARN-2674.3.patch with a test case and more

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571957#comment-14571957 ] Chun Chen commented on YARN-3749: - Thanks for reviewing and committing the patch, [~xgong].

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14570202#comment-14570202 ] Chun Chen commented on YARN-3749: - Thanks for reviewing the patch, [~zxu] ! We should

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568586#comment-14568586 ] Chun Chen commented on YARN-3749: - bq. It looks like we need keep

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.2.patch Upload a new patch to fix test cases. Lots of previous tests The HA

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.7.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.4.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.5.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568521#comment-14568521 ] Chun Chen commented on YARN-3749: - [~zxu] ,thanks, I agree. Upload YARN-3749.6.patch to

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.3.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568448#comment-14568448 ] Chun Chen commented on YARN-3749: - Thanks for the review [~zxu] [~iwasakims]. Upload a new

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.6.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568458#comment-14568458 ] Chun Chen commented on YARN-3749: - Upload a YARN-3749.5.patch to set

[jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566326#comment-14566326 ] Chun Chen commented on YARN-3749: - Upload a patch to fix it. We should make a copy of

[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566327#comment-14566327 ] Chun Chen commented on YARN-2674: - Thanks for the comments [~vinodkv]. Will upload a new

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Summary: We should make a copy of configuration when init MiniYARNCluster with multiple RMs (was: We should

[jira] [Created] (YARN-3749) We should make a copy of config MiniYARNCluster

Chun Chen created YARN-3749: --- Summary: We should make a copy of config MiniYARNCluster Key: YARN-3749 URL: https://issues.apache.org/jira/browse/YARN-3749 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Description: When I was trying to write a test case for YARN-2674, I found DS client trying to connect to

[jira] [Assigned] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen reassigned YARN-3749: --- Assignee: Chun Chen We should make a copy of configuration when init MiniYARNCluster with multiple

[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs

[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Attachment: YARN-3749.patch We should make a copy of configuration when init MiniYARNCluster with

[jira] [Commented] (YARN-3080) The DockerContainerExecutor could not write the right pid to container pidFile

[ https://issues.apache.org/jira/browse/YARN-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341427#comment-14341427 ] Chun Chen commented on YARN-3080: - [~ashahab], I think we can simply fix this by using the

[jira] [Commented] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307247#comment-14307247 ] Chun Chen commented on YARN-1983: - Thanks for the comments, [~vinodkv] [~chris.douglas]. Do

[jira] [Updated] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-1983: Attachment: YARN-1983.2.patch Update the patch to rewrite unit tests. Support heterogeneous container

[jira] [Commented] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296263#comment-14296263 ] Chun Chen commented on YARN-3077: - [~ozawa], OK, upload a new patch to update the patch and

[jira] [Updated] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3077: Attachment: YARN-3077.3.patch RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[jira] [Commented] (YARN-2718) Create a CompositeConatainerExecutor that combines DockerContainerExecutor and DefaultContainerExecutor

[ https://issues.apache.org/jira/browse/YARN-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293050#comment-14293050 ] Chun Chen commented on YARN-2718: - Yes, my mistake. Thanks for pointing out. Attach a fixed

[jira] [Updated] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-1983: Attachment: YARN-1983.patch Support heterogeneous container types at runtime on YARN

[jira] [Commented] (YARN-2992) ZKRMStateStore crashes due to session expiry

[ https://issues.apache.org/jira/browse/YARN-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292937#comment-14292937 ] Chun Chen commented on YARN-2992: - [~kasha] [~rohithsharma] [~jianhe], we are constantly

[jira] [Commented] (YARN-2718) Create a CompositeConatainerExecutor that combines DockerContainerExecutor and DefaultContainerExecutor

[ https://issues.apache.org/jira/browse/YARN-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292903#comment-14292903 ] Chun Chen commented on YARN-2718: - Well, The patch I've uploaded mainly focuses on making

[jira] [Commented] (YARN-1983) Support heterogeneous container types at runtime on YARN

[ https://issues.apache.org/jira/browse/YARN-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293060#comment-14293060 ] Chun Chen commented on YARN-1983: - Attach a patch which creates a

[jira] [Commented] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290363#comment-14290363 ] Chun Chen commented on YARN-3077: - Thanks for reviewing the patch, [~jianhe]. upload a new

[jira] [Updated] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3077: Attachment: YARN-3077.2.patch RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[jira] [Commented] (YARN-3094) reset timer for liveness monitors after RM recovery

[ https://issues.apache.org/jira/browse/YARN-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290395#comment-14290395 ] Chun Chen commented on YARN-3094: - Since RM can't receive ping from AM util

[jira] [Commented] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285419#comment-14285419 ] Chun Chen commented on YARN-3077: - The failed tests passed in my owner laptop. RM should

[jira] [Commented] (YARN-2466) Umbrella issue for Yarn launched Docker Containers

[ https://issues.apache.org/jira/browse/YARN-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285434#comment-14285434 ] Chun Chen commented on YARN-2466: - I think this feature is currently only a alpha version

[jira] [Updated] (YARN-2718) Create a CompositeConatainerExecutor that combines DockerContainerExecutor and DefaultContainerExecutor

[ https://issues.apache.org/jira/browse/YARN-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2718: Attachment: YARN-2718.patch Upload my patch to share my thoughts about creating a

[jira] [Commented] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285174#comment-14285174 ] Chun Chen commented on YARN-3077: - Thanks. :) RM should create

[jira] [Commented] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285161#comment-14285161 ] Chun Chen commented on YARN-3077: - [~varun_saxena] I would like to do it myself. RM

[jira] [Created] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

Chun Chen created YARN-3077: --- Summary: RM should create yarn.resourcemanager.zk-state-store.parent-path recursively Key: YARN-3077 URL: https://issues.apache.org/jira/browse/YARN-3077 Project: Hadoop YARN

[jira] [Updated] (YARN-3077) RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[ https://issues.apache.org/jira/browse/YARN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3077: Attachment: YARN-3077.patch RM should create yarn.resourcemanager.zk-state-store.parent-path recursively

[jira] [Commented] (YARN-2262) Few fields displaying wrong values in Timeline server after RM restart

[ https://issues.apache.org/jira/browse/YARN-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282328#comment-14282328 ] Chun Chen commented on YARN-2262: - I think the second exception is because TFile is

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Attachment: YARN-2674.2.patch Distributed shell AM may re-launch containers if RM work preserving restart

[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231719#comment-14231719 ] Chun Chen commented on YARN-2674: - Thanks for review, [~jianhe] , upload a new patch

[jira] [Commented] (YARN-2718) Create a CompositeConatainerExecutor that combines DockerContainerExecutor and DefaultContainerExecutor

[ https://issues.apache.org/jira/browse/YARN-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214690#comment-14214690 ] Chun Chen commented on YARN-2718: - [~ashahab], if you don't mind, can I work on this. I

[jira] [Commented] (YARN-1964) Create Docker analog of the LinuxContainerExecutor in YARN

[ https://issues.apache.org/jira/browse/YARN-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201527#comment-14201527 ] Chun Chen commented on YARN-1964: - Hi, [~ashahab], thanks for the patch. I’m also working

[jira] [Created] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

Chun Chen created YARN-2674: --- Summary: Distributed shell AM may re-launch containers if RM work preserving restart happens Key: YARN-2674 URL: https://issues.apache.org/jira/browse/YARN-2674 Project:

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Attachment: YARN-2674.1.patch Distributed shell AM may re-launch containers if RM work preserving restart

[jira] [Updated] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens

[ https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2674: Description: Currently, if RM work preserving restart happens while distributed shell is running,

[jira] [Updated] (YARN-2612) Some completed containers are not reported to NM

[ https://issues.apache.org/jira/browse/YARN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-2612: Description: We are testing RM work preserving restart and found the following logs when we ran a simple