[jira] [Commented] (YARN-9560) Restructure DockerLinuxContainerRuntime to extend a new OCIContainerRuntime

2019-05-21 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844919#comment-16844919 ] Jim Brennan commented on YARN-9560: --- Thanks for the updates [~ebadger]! Some comments on the new patch:

[jira] [Commented] (YARN-9561) Add C changes for the new OCI/squashfs/runc runtime

2019-05-20 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844311#comment-16844311 ] Jim Brennan commented on YARN-9561: --- [~ebadger] thanks for the patch!  I did an extensive review of this

[jira] [Commented] (YARN-9560) Restructure DockerLinuxContainerRuntime to extend a new OCIContainerRuntime

2019-05-20 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844121#comment-16844121 ] Jim Brennan commented on YARN-9560: --- [~ebadger] Thanks for the patch. Overall this looks like a good

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2019-05-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839719#comment-16839719 ] Jim Brennan commented on YARN-2194: --- As I mentioned in [YARN-9518], I am also +1 on this patch for

[jira] [Commented] (YARN-9518) can't use CGroups with YARN in centos7

2019-05-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839693#comment-16839693 ] Jim Brennan commented on YARN-9518: --- Thanks [~jhung]!  The patch looks good to me.  I also built it and

[jira] [Commented] (YARN-9518) can't use CGroups with YARN in centos7

2019-05-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839570#comment-16839570 ] Jim Brennan commented on YARN-9518: --- [~shurong.mai], ideally we would use the same solution in 2.7 that

[jira] [Commented] (YARN-9518) can't use CGroups with YARN in centos7

2019-05-09 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836669#comment-16836669 ] Jim Brennan commented on YARN-9518: --- [~shurong.mai], i believe that [~jhung] is correct, and this

[jira] [Commented] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-09 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836638#comment-16836638 ] Jim Brennan commented on YARN-9527: --- I was able to repro the problem in branch-2.8 on a one-node-cluster

[jira] [Commented] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-09 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836456#comment-16836456 ] Jim Brennan commented on YARN-9527: --- Thanks for the review [~ebadger]!  I've put up another patch that

[jira] [Updated] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-09 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9527: -- Attachment: YARN-9527.004.patch > Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same

[jira] [Commented] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835623#comment-16835623 ] Jim Brennan commented on YARN-9527: --- I put up patch 003 to address the checkstyle issues.   > Rogue

[jira] [Updated] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9527: -- Attachment: YARN-9527.003.patch > Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same

[jira] [Updated] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-07 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9527: -- Attachment: YARN-9527.002.patch > Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same

[jira] [Assigned] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-03 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reassigned YARN-9527: - Assignee: Jim Brennan > Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same

[jira] [Commented] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-02 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831903#comment-16831903 ] Jim Brennan commented on YARN-9527: --- I was able to find a node where the problem was actively happening,

[jira] [Commented] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-02 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831883#comment-16831883 ] Jim Brennan commented on YARN-9527: --- For example, we recently had a case where all of the disks used by

[jira] [Created] (YARN-9527) Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file

2019-05-02 Thread Jim Brennan (JIRA)
Jim Brennan created YARN-9527: - Summary: Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file Key: YARN-9527 URL: https://issues.apache.org/jira/browse/YARN-9527 Project: Hadoop YARN

[jira] [Commented] (YARN-9518) can't use CGroups with YARN in centos7

2019-04-30 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830347#comment-16830347 ] Jim Brennan commented on YARN-9518: --- [~shurong.mai], your patch needs to be based on branch trunk.   I

[jira] [Commented] (YARN-9518) can't use CGroups with YARN in centos7

2019-04-29 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829328#comment-16829328 ] Jim Brennan commented on YARN-9518: --- [~shurong.mai], are you running with the latest code (trunk)?   The

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826198#comment-16826198 ] Jim Brennan commented on YARN-9486: --- [~eyang] thanks for updating the comment.  +1 (non-binding) on

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826142#comment-16826142 ] Jim Brennan commented on YARN-9486: --- [~eyang], I am +1 (non-binding) on patch 004. > Docker container

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826101#comment-16826101 ] Jim Brennan commented on YARN-9486: --- {quote} As the result, we need to check both markedLaunched and

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-24 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825465#comment-16825465 ] Jim Brennan commented on YARN-9486: --- {quote}Patch 003 added the safe guard for missing pid file, and

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-24 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825404#comment-16825404 ] Jim Brennan commented on YARN-9486: --- [~eyang] {quote} The right logic is probably try to locate it

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-24 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825333#comment-16825333 ] Jim Brennan commented on YARN-9486: --- [~eyang] I am not too familiar with the ContainerRelaunch path, but

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-24 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825222#comment-16825222 ] Jim Brennan commented on YARN-9486: --- [~eyang] I'm not sure I agree. This suggests that

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-22 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823309#comment-16823309 ] Jim Brennan commented on YARN-9486: --- {quote} It looks like the problem is in the usage of

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-22 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823189#comment-16823189 ] Jim Brennan commented on YARN-9486: --- [~eyang] I'm just trying to understand the logic here.

[jira] [Commented] (YARN-9486) Docker container exited with failure does not get clean up correctly

2019-04-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822028#comment-16822028 ] Jim Brennan commented on YARN-9486: --- [~eyang], why is launch.markLaunched() returning false in this

[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try

2019-04-11 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815434#comment-16815434 ] Jim Brennan commented on YARN-7848: --- Thanks for updating [~eyang]!  lgtm.   I am +1 on patch 004

[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try

2019-04-10 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814627#comment-16814627 ] Jim Brennan commented on YARN-7848: --- Thanks [~eyang] for the update! Some comments: In

[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try

2019-04-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812744#comment-16812744 ] Jim Brennan commented on YARN-7848: --- [~eyang], [~ebadger], it looks to me like the code in patch 002

[jira] [Commented] (YARN-9442) container working directory has group read permissions

2019-04-05 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811131#comment-16811131 ] Jim Brennan commented on YARN-9442: --- [~eyang] thanks for testing!  I also ran terasort as part of my

[jira] [Commented] (YARN-9442) container working directory has group read permissions

2019-04-05 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811021#comment-16811021 ] Jim Brennan commented on YARN-9442: --- {quote} I think the group read permission is used by shuffle

[jira] [Commented] (YARN-9442) container working directory has group read permissions

2019-04-05 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811011#comment-16811011 ] Jim Brennan commented on YARN-9442: --- I have put up patch.002 which adds a check to the 

[jira] [Updated] (YARN-9442) container working directory has group read permissions

2019-04-05 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9442: -- Attachment: YARN-9442.002.patch > container working directory has group read permissions >

[jira] [Comment Edited] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809950#comment-16809950 ] Jim Brennan edited comment on YARN-9442 at 4/4/19 3:25 PM: --- Specifically, we

[jira] [Comment Edited] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809950#comment-16809950 ] Jim Brennan edited comment on YARN-9442 at 4/4/19 3:23 PM: --- Specifically, we

[jira] [Assigned] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reassigned YARN-9442: - Assignee: Jim Brennan > container working directory has group read permissions >

[jira] [Commented] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809950#comment-16809950 ] Jim Brennan commented on YARN-9442: --- Specifically, we have something like this now: {noformat}

[jira] [Updated] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9442: -- Priority: Minor (was: Major) > container working directory has group read permissions >

[jira] [Created] (YARN-9442) container working directory has group read permissions

2019-04-04 Thread Jim Brennan (JIRA)
Jim Brennan created YARN-9442: - Summary: container working directory has group read permissions Key: YARN-9442 URL: https://issues.apache.org/jira/browse/YARN-9442 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try

2019-03-27 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802833#comment-16802833 ] Jim Brennan commented on YARN-7848: --- I think I agree with [~eyang] on the use of {{-f}}.  By the time we

[jira] [Commented] (YARN-8200) Backport resource types/GPU features to branch-3.0/branch-2

2019-03-26 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801916#comment-16801916 ] Jim Brennan commented on YARN-8200: --- Thanks [~jhung].   Is the intention to propose a 2.10 release with

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800957#comment-16800957 ] Jim Brennan commented on YARN-9391: --- [~ebadger], [~eyang] patch 001 is ready for review.   > Disable

[jira] [Updated] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9391: -- Attachment: YARN-9391.001.patch > Disable PATH variable to be passed to Docker container >

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-20 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797182#comment-16797182 ] Jim Brennan commented on YARN-9391: --- OK.  I will put up a patch to fix this issue.   > Disable PATH

[jira] [Assigned] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-20 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reassigned YARN-9391: - Assignee: Jim Brennan > Disable PATH variable to be passed to Docker container >

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796597#comment-16796597 ] Jim Brennan commented on YARN-9391: --- [~eyang] if the concern is only for Nodemanager white-list

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796560#comment-16796560 ] Jim Brennan commented on YARN-9391: --- [~ebadger] I assume you are referring to the PATH variable in

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796521#comment-16796521 ] Jim Brennan commented on YARN-9391: --- {quote} The whitelist needs to behave differently for docker

[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container

2019-03-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796227#comment-16796227 ] Jim Brennan commented on YARN-9391: --- [~ebadger] you are correct.  It doesn't look like we explicitly add

[jira] [Commented] (YARN-8200) Backport resource types/GPU features to branch-3.0/branch-2

2019-03-15 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793917#comment-16793917 ] Jim Brennan commented on YARN-8200: --- [~jhung] what is the status of this effort?  Are you still working

[jira] [Commented] (YARN-8376) Separate white list for docker.trusted.registries and docker.privileged-container.registries

2019-03-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792661#comment-16792661 ] Jim Brennan commented on YARN-8376: --- +1 on patch 003 (non-binding)   > Separate white list for

[jira] [Commented] (YARN-8376) Separate white list for docker.trusted.registries and docker.privileged-container.registries

2019-03-13 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792100#comment-16792100 ] Jim Brennan commented on YARN-8376: --- To be clear, I am ok with the approach used in patch 002, where

[jira] [Commented] (YARN-8376) Separate white list for docker.trusted.registries and docker.privileged-container.registries

2019-03-13 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791855#comment-16791855 ] Jim Brennan commented on YARN-8376: --- [~eyang] [~ebadger] sorry for joining this late.  It seems like the

[jira] [Commented] (YARN-7129) Application Catalog for YARN applications

2019-03-12 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791024#comment-16791024 ] Jim Brennan commented on YARN-7129: --- {quote}Now, Hadoop build process intended to introduce inline

[jira] [Commented] (YARN-7129) Application Catalog for YARN applications

2019-03-12 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790838#comment-16790838 ] Jim Brennan commented on YARN-7129: --- I agree with [~ebadger].  Suddenly today I am unable to build

[jira] [Commented] (YARN-5714) ContainerExecutor does not order environment map

2019-03-06 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786121#comment-16786121 ] Jim Brennan commented on YARN-5714: --- Thanks [~eepayne]!  I re-uploaded the branch-2 patch.   >

[jira] [Updated] (YARN-5714) ContainerExecutor does not order environment map

2019-03-06 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-5714: -- Attachment: (was: YARN-5714-branch-2.001.patch) > ContainerExecutor does not order environment map

[jira] [Updated] (YARN-5714) ContainerExecutor does not order environment map

2019-03-06 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-5714: -- Attachment: YARN-5714-branch-2.001.patch > ContainerExecutor does not order environment map >

[jira] [Assigned] (YARN-5714) ContainerExecutor does not order environment map

2019-03-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reassigned YARN-5714: - Assignee: Remi Catherinot (was: Jim Brennan) > ContainerExecutor does not order environment map

[jira] [Commented] (YARN-5714) ContainerExecutor does not order environment map

2019-03-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783437#comment-16783437 ] Jim Brennan commented on YARN-5714: --- cc: [~eepayne] I changed the assignee back to [~rcatherinot] - I

[jira] [Updated] (YARN-5714) ContainerExecutor does not order environment map

2019-03-01 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-5714: -- Attachment: YARN-5714-branch-2.8.001.patch YARN-5714-branch-2.001.patch >

[jira] [Reopened] (YARN-5714) ContainerExecutor does not order environment map

2019-03-01 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan reopened YARN-5714: --- Assignee: Jim Brennan (was: Remi Catherinot) Re-opening so I can put up branch-2 patches. >

[jira] [Commented] (YARN-9098) Separate mtab file reader code and cgroups file system hierarchy parser code from CGroupsHandlerImpl and ResourceHandlerModule

2019-02-25 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777136#comment-16777136 ] Jim Brennan commented on YARN-9098: --- I downloaded patch 007, built it and ran unit tests. +1 this looks

[jira] [Commented] (YARN-9098) Separate mtab file reader code and cgroups file system hierarchy parser code from CGroupsHandlerImpl and ResourceHandlerModule

2019-02-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763695#comment-16763695 ] Jim Brennan commented on YARN-9098: --- [~snemeth] thanks for updating the patch!   I agree with your fix

[jira] [Commented] (YARN-9206) RMServerUtils does not count SHUTDOWN as an accepted state

2019-02-07 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762738#comment-16762738 ] Jim Brennan commented on YARN-9206: --- The new patches look good to me.   > RMServerUtils does not count

[jira] [Commented] (YARN-9206) RMServerUtils does not count SHUTDOWN as an accepted state

2019-01-31 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757718#comment-16757718 ] Jim Brennan commented on YARN-9206: --- [~kshukla] thanks for the new patch.   This looks good to me.  +1

[jira] [Commented] (YARN-9206) RMServerUtils does not count SHUTDOWN as an accepted state

2019-01-24 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751227#comment-16751227 ] Jim Brennan commented on YARN-9206: --- [~sunilg], [~kshukla]  While I agree that [~sunilg]'s version looks

[jira] [Commented] (YARN-9206) RMServerUtils does not count SHUTDOWN as an accepted state

2019-01-22 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749132#comment-16749132 ] Jim Brennan commented on YARN-9206: --- {quote}{quote}I think you need to iterate the acceptedStates() and

[jira] [Commented] (YARN-9206) RMServerUtils does not count SHUTDOWN as an accepted state

2019-01-22 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748895#comment-16748895 ] Jim Brennan commented on YARN-9206: --- [~kshukla], thanks for the patch!  I think it would be cleaner if

[jira] [Commented] (YARN-9003) Support multi-homed network for docker container

2019-01-18 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746448#comment-16746448 ] Jim Brennan commented on YARN-9003: --- [~eyang], {quote} Docker does not support multiple network

[jira] [Commented] (YARN-9202) RM does not track nodes that are in the include list and never register

2019-01-17 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745516#comment-16745516 ] Jim Brennan commented on YARN-9202: --- [~kshukla] thanks for the patch! I'd like some clarification of

[jira] [Commented] (YARN-9003) Support multi-homed network for docker container

2019-01-16 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744202#comment-16744202 ] Jim Brennan commented on YARN-9003: --- [~eyang] thanks for the patch.  I have a few comments on the code:

[jira] [Commented] (YARN-9053) Support set environment variables for Docker Containers In nonEntryPoint mode

2019-01-15 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743217#comment-16743217 ] Jim Brennan commented on YARN-9053: --- [~Charo Zhang], [~eyang] Can we close this Jira as Not-Doing? >

[jira] [Commented] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally

2019-01-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742429#comment-16742429 ] Jim Brennan commented on YARN-9184: --- I also agree with [~ebadger] and [~eyang] that this should be a

[jira] [Commented] (YARN-9074) Docker container rm command should be executed after stop

2019-01-14 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742256#comment-16742256 ] Jim Brennan commented on YARN-9074: --- +1 (non-binding) this looks good to me.   > Docker container rm

[jira] [Commented] (YARN-9098) Separate mtab file reader code and cgroups file system hierarchy parser code from CGroupsHandlerImpl and ResourceHandlerModule

2018-12-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725212#comment-16725212 ] Jim Brennan commented on YARN-9098: --- [~snemeth] I found the bug (or at least one bug). In

[jira] [Commented] (YARN-9098) Separate mtab file reader code and cgroups file system hierarchy parser code from CGroupsHandlerImpl and ResourceHandlerModule

2018-12-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725136#comment-16725136 ] Jim Brennan commented on YARN-9098: --- [~snemeth] thanks for updating the patch.  I will download and

[jira] [Commented] (YARN-9098) Separate mtab file reader code and cgroups file system hierarchy parser code from CGroupsHandlerImpl and ResourceHandlerModule

2018-12-18 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724524#comment-16724524 ] Jim Brennan commented on YARN-9098: --- [~snemeth] thanks for the patch! In general the code looks good to

[jira] [Commented] (YARN-9053) Support set environment variables for Docker Containers In nonEntryPoint mode

2018-11-29 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703337#comment-16703337 ] Jim Brennan commented on YARN-9053: --- [~Charo Zhang] can you explain more about what your users are doing

[jira] [Commented] (YARN-9053) Support set environment variables for Docker Containers In nonEntryPoint mode

2018-11-28 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701974#comment-16701974 ] Jim Brennan commented on YARN-9053: --- The launch_script.sh is run inside the docker container, so any

[jira] [Commented] (YARN-9053) Support set environment variables for Docker Containers In nonEntryPoint mode

2018-11-27 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700556#comment-16700556 ] Jim Brennan commented on YARN-9053: --- {quote} Besides, we don't think white-listed way is good, because

[jira] [Commented] (YARN-9053) Support set environment variables for Docker Containers In nonEntryPoint mode

2018-11-26 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699142#comment-16699142 ] Jim Brennan commented on YARN-9053: --- {quote} Environment variables are not written to docker container.

[jira] [Commented] (YARN-8911) NM incorrectly account for container cpu utilization by their number of vcores

2018-10-19 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657448#comment-16657448 ] Jim Brennan commented on YARN-8911: --- [~haibochen], can you provide a little more detail about what lead

[jira] [Commented] (YARN-6091) the AppMaster register failed when use Docker on LinuxContainer

2018-10-02 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636178#comment-16636178 ] Jim Brennan commented on YARN-6091: --- [~jlowe], [~ebadger] this does indeed look like it was fixed by

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-13 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613729#comment-16613729 ] Jim Brennan commented on YARN-8648: --- [~jlowe] thanks for the review.   I've made the change you

[jira] [Updated] (YARN-8648) Container cgroups are leaked when using docker

2018-09-13 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-8648: -- Attachment: YARN-8648.006.patch > Container cgroups are leaked when using docker >

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-12 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612556#comment-16612556 ] Jim Brennan commented on YARN-8648: --- Thanks [~jlowe] for the review!  I have addressed all of this

[jira] [Updated] (YARN-8648) Container cgroups are leaked when using docker

2018-09-12 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-8648: -- Attachment: YARN-8648.005.patch > Container cgroups are leaked when using docker >

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-10 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609206#comment-16609206 ] Jim Brennan commented on YARN-8648: --- This is ready for review.   > Container cgroups are leaked when

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608212#comment-16608212 ] Jim Brennan commented on YARN-8648: --- I have uploaded a patch that adds the cgroup cleanup to the

[jira] [Updated] (YARN-8648) Container cgroups are leaked when using docker

2018-09-08 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-8648: -- Attachment: YARN-8648.004.patch > Container cgroups are leaked when using docker >

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603606#comment-16603606 ] Jim Brennan commented on YARN-8648: --- Put up another patch to fix the checkstyle issue.   > Container

[jira] [Updated] (YARN-8648) Container cgroups are leaked when using docker

2018-09-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-8648: -- Attachment: YARN-8648.003.patch > Container cgroups are leaked when using docker >

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-09-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603202#comment-16603202 ] Jim Brennan commented on YARN-8648: --- I've uploaded a patch that addresses most of issues raised by

[jira] [Updated] (YARN-8648) Container cgroups are leaked when using docker

2018-09-04 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-8648: -- Attachment: YARN-8648.002.patch > Container cgroups are leaked when using docker >

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-08-31 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599235#comment-16599235 ] Jim Brennan commented on YARN-8648: --- {quote} I explicitly check that the directory exists before calling

[jira] [Commented] (YARN-8648) Container cgroups are leaked when using docker

2018-08-31 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599213#comment-16599213 ] Jim Brennan commented on YARN-8648: --- [~jlowe] thanks for the review! {quote}Why was the postComplete

<    1   2   3   4   5   6   7   8   9   >