[jira] [Updated] (YARN-8811) Support Container Storage Interface (CSI) in YARN

2018-09-21 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated YARN-8811: -- Attachment: Support Container Storage Interface(CSI) in YARN_design doc_20180921.pdf > Support

[jira] [Commented] (YARN-8794) QueuePlacementPolicy add more rules

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623249#comment-16623249 ] Hadoop QA commented on YARN-8794: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (YARN-8811) Support Container Storage Interface (CSI) in YARN

2018-09-21 Thread Sunil Govindan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan reassigned YARN-8811: Assignee: Weiwei Yang > Support Container Storage Interface (CSI) in YARN >

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.003.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Bibin A Chundatt (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623321#comment-16623321 ] Bibin A Chundatt commented on YARN-7599: Thank you [~botong] Over all patch looks good to me..

[jira] [Comment Edited] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-21 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623329#comment-16623329 ] Antal Bálint Steinbach edited comment on YARN-8759 at 9/21/18 9:31 AM:

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623301#comment-16623301 ] Hadoop QA commented on YARN-8806: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-21 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8759: - Attachment: YARN-8759.004.patch > Copy of "resource-types.xml" is not deleted if

[jira] [Comment Edited] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application

2018-09-21 Thread Akhil PB (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623380#comment-16623380 ] Akhil PB edited comment on YARN-7957 at 9/21/18 10:29 AM: -- Below is the updated

[jira] [Comment Edited] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application

2018-09-21 Thread Akhil PB (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623380#comment-16623380 ] Akhil PB edited comment on YARN-7957 at 9/21/18 10:29 AM: -- Below is the updated

[jira] [Commented] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623412#comment-16623412 ] Hadoop QA commented on YARN-8812: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-2599) Standby RM should also expose some jmx and metrics

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623315#comment-16623315 ] Tao Yang commented on YARN-2599: Hi, [~Naganarasimha]. Some JVM metrics of standby RM can be evaluated

[jira] [Created] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Oleksandr Shevchenko (JIRA)
Oleksandr Shevchenko created YARN-8812: -- Summary: Containers fail during creating a symlink which started with hyphen for a resource file Key: YARN-8812 URL: https://issues.apache.org/jira/browse/YARN-8812

[jira] [Comment Edited] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application

2018-09-21 Thread Akhil PB (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623380#comment-16623380 ] Akhil PB edited comment on YARN-7957 at 9/21/18 10:26 AM: -- Hi [~sunilg] Below is

[jira] [Commented] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Oleksandr Shevchenko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623324#comment-16623324 ] Oleksandr Shevchenko commented on YARN-8812: Changed generating symlink creating command by

[jira] [Comment Edited] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Oleksandr Shevchenko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623439#comment-16623439 ] Oleksandr Shevchenko edited comment on YARN-8812 at 9/21/18 11:38 AM: --

[jira] [Commented] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-09-21 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623426#comment-16623426 ] Weiwei Yang commented on YARN-8468: --- Thanks [~bsteinbach] for the updates. One thing I am not sure, as 

[jira] [Commented] (YARN-8628) [UI2] Few duplicated or inconsistent information displayed in UI2

2018-09-21 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623416#comment-16623416 ] Hudson commented on YARN-8628: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15034 (See

[jira] [Commented] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Oleksandr Shevchenko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623439#comment-16623439 ] Oleksandr Shevchenko commented on YARN-8812: Test Proxy#testNMProxyRPCRetry failed by a

[jira] [Commented] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623565#comment-16623565 ] Hadoop QA commented on YARN-8759: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623494#comment-16623494 ] Zhankun Tang commented on YARN-8806: [~yuan_zac] Thanks for providing the patch. I also noticed this

[jira] [Comment Edited] (YARN-8812) Containers fail during creating a symlink which started with hyphen for a resource file

2018-09-21 Thread Oleksandr Shevchenko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623439#comment-16623439 ] Oleksandr Shevchenko edited comment on YARN-8812 at 9/21/18 11:48 AM: --

[jira] [Commented] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-21 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623329#comment-16623329 ] Antal Bálint Steinbach commented on YARN-8759: -- Hi [~maniraj...@gmail.com] , Unfortunately

[jira] [Commented] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-21 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623356#comment-16623356 ] Manikandan R commented on YARN-8759: {quote}If you think it is confusing I can simply remove that

[jira] [Commented] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application

2018-09-21 Thread Akhil PB (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623380#comment-16623380 ] Akhil PB commented on YARN-7957: Hi [~sunilg] Below is the updated service states. {code:java} public

[jira] [Comment Edited] (YARN-7957) [UI2] Yarn service delete option disappears after stopping application

2018-09-21 Thread Akhil PB (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623380#comment-16623380 ] Akhil PB edited comment on YARN-7957 at 9/21/18 10:25 AM: -- Hi [~sunilg] Below is

[jira] [Commented] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-09-21 Thread Xun Liu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623763#comment-16623763 ] Xun Liu commented on YARN-8563: --- I don't think submarine needs to care about image management and production

[jira] [Updated] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-21 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated YARN-8789: -- Attachment: YARN-8789.4.patch > Add BoundedQueue to AsyncDispatcher >

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-8804: - Target Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2, 2.8.6 Thanks for updating the patch! This is a

[jira] [Commented] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-09-21 Thread Xun Liu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623762#comment-16623762 ] Xun Liu commented on YARN-8563: --- I don't think submarine needs to care about image management and production

[jira] [Issue Comment Deleted] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-09-21 Thread Xun Liu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu updated YARN-8563: -- Comment: was deleted (was: I don't think submarine needs to care about image management and production issues

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623622#comment-16623622 ] ASF GitHub Bot commented on YARN-8785: -- GitHub user simonprewo opened a pull request:

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623619#comment-16623619 ] Zac Zhou commented on YARN-8806: [~tangzhankun] Thanks a lot for your suggestion. :) I thought your idea

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-21 Thread Simon Prewo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623648#comment-16623648 ] Simon Prewo commented on YARN-8785: --- [~Zian Chen] Thanks for your support - feel free to review and give

[jira] [Commented] (YARN-8777) Container Executor C binary change to execute interactive docker command

2018-09-21 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623675#comment-16623675 ] Eric Badger commented on YARN-8777: --- Ok I see what you're saying. I'm fine with the modifications to

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-21 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624007#comment-16624007 ] Zian Chen commented on YARN-8785: - Hi [~simonprewo], thanks for the patch. The patch itself looks good to

[jira] [Updated] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7599: --- Attachment: YARN-7599-YARN-7402.v8.patch > [GPG] ApplicationCleaner in Global Policy Generator >

[jira] [Commented] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624091#comment-16624091 ] Hadoop QA commented on YARN-8789: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624250#comment-16624250 ] Gour Saha commented on YARN-8734: - [~eyang] this is a pretty useful feature so thanks for taking this up.

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624281#comment-16624281 ] Tao Yang commented on YARN-8804: Thanks [~jlowe] for the review. Attached v3 patch to rebase it on latest

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623994#comment-16623994 ] Hadoop QA commented on YARN-8808: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624224#comment-16624224 ] Arun Suresh edited comment on YARN-8808 at 9/21/18 9:50 PM: While working on

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Commented] (YARN-8769) [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job

2018-09-21 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624018#comment-16624018 ] Hudson commented on YARN-8769: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15035 (See

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624015#comment-16624015 ] Haibo Chen commented on YARN-8808: -- The unit test failure is independent of this patch. > Use aggregate

[jira] [Updated] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-21 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated YARN-8789: -- Attachment: YARN-8789.5.patch > Add BoundedQueue to AsyncDispatcher >

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: (was: YARN-8774.branch-2.001.patch) > Memory leak when CapacityScheduler allocates from

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.8.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2018-09-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624032#comment-16624032 ] Arun Suresh commented on YARN-1011: --- I was trying to rebase the branch with trunk.. Got a couple of

[jira] [Updated] (YARN-8809) Refactor AbstractYarnScheduler and CapacityScheduler OPPORTUNISTIC container completion codepaths

2018-09-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-8809: -- Summary: Refactor AbstractYarnScheduler and CapacityScheduler OPPORTUNISTIC container completion

[jira] [Updated] (YARN-6510) Fix profs stat file warning caused by process names that includes parenthesis

2018-09-21 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-6510: - Fix Version/s: 2.8.6 Thanks, [~wilfreds]! I committed this to branch-2.8 as well. > Fix profs stat file

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623985#comment-16623985 ] Wangda Tan commented on YARN-8806: -- [~yuan_zac], I think we should create a process when submit jobs.

[jira] [Commented] (YARN-6456) Allow administrators to set a single ContainerRuntime for all containers

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624009#comment-16624009 ] Hadoop QA commented on YARN-6456: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624029#comment-16624029 ] Arun Suresh commented on YARN-8809: --- Thanks [~haibochen] +1, will commit shortly > Fair Scheduler does

[jira] [Updated] (YARN-6456) Allow administrators to set a single ContainerRuntime for all containers

2018-09-21 Thread Craig Condit (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Condit updated YARN-6456: --- Attachment: (was: YARN-6456.005.patch) > Allow administrators to set a single ContainerRuntime

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Attachment: YARN-8804.003.patch > resourceLimits may be wrongly calculated when leaf-queue is blocked in >

[jira] [Comment Edited] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624340#comment-16624340 ] Eric Yang edited comment on YARN-8734 at 9/21/18 11:54 PM: --- {quote}In that case

[jira] [Commented] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624340#comment-16624340 ] Eric Yang commented on YARN-8734: - {quote}In that case may be a simpler approach will be to call this

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: (was: YARN-8774.branch-2.8.001.patch) > Memory leak when CapacityScheduler allocates from

[jira] [Commented] (YARN-8777) Container Executor C binary change to execute interactive docker command

2018-09-21 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623999#comment-16623999 ] Zian Chen commented on YARN-8777: - Thanks [~eyang] for the work. I'm ok with patch 003. One quick

[jira] [Created] (YARN-8814) Yarn Service Upgrade: Update the swagger definition and docs

2018-09-21 Thread Chandni Singh (JIRA)
Chandni Singh created YARN-8814: --- Summary: Yarn Service Upgrade: Update the swagger definition and docs Key: YARN-8814 URL: https://issues.apache.org/jira/browse/YARN-8814 Project: Hadoop YARN

[jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624053#comment-16624053 ] Botong Huang commented on YARN-7599: Ah good point. v8 uploaded. Will commit pending on yetus. Thanks!

[jira] [Commented] (YARN-8813) Improve debug messages for NM preemption of OPPORTUNISTIC containers

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624089#comment-16624089 ] Hadoop QA commented on YARN-8813: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (YARN-8623) Update Docker examples to use image which exists

2018-09-21 Thread Craig Condit (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Condit reassigned YARN-8623: -- Assignee: Craig Condit > Update Docker examples to use image which exists >

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624109#comment-16624109 ] Wangda Tan commented on YARN-8785: -- [~simonprewo], instead of uploading patch file, you can rename the

[jira] [Commented] (YARN-8623) Update Docker examples to use image which exists

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624175#comment-16624175 ] Hadoop QA commented on YARN-8623: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-21 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624224#comment-16624224 ] Arun Suresh commented on YARN-8808: --- While working on YARN-1013, looks like we should also check for

[jira] [Commented] (YARN-8696) [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624248#comment-16624248 ] Hadoop QA commented on YARN-8696: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624306#comment-16624306 ] Eric Yang commented on YARN-8734: - [~gsaha] Local service would imply the current service itself in my

[jira] [Commented] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624319#comment-16624319 ] Gour Saha commented on YARN-8734: - In that case may be a simpler approach will be to call this property

[jira] [Commented] (YARN-8809) Refactor AbstractYarnScheduler and CapacityScheduler OPPORTUNISTIC container completion codepaths

2018-09-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624169#comment-16624169 ] Haibo Chen commented on YARN-8809: -- Thanks for the review and commit, [~asuresh] > Refactor

[jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624130#comment-16624130 ] Hadoop QA commented on YARN-7599: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.002.patch > Memory leak when CapacityScheduler allocates from reserved container with

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Updated] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8774: --- Attachment: YARN-8774.branch-2.8.001.patch > Memory leak when CapacityScheduler allocates from reserved

[jira] [Commented] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624379#comment-16624379 ] Hadoop QA commented on YARN-8789: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624367#comment-16624367 ] Tao Yang edited comment on YARN-8774 at 9/22/18 12:17 AM: -- Thanks [~eepayne] for

[jira] [Updated] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-21 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-8665: Attachment: YARN-8665.004.patch > Yarn Service Upgrade: Support cancelling upgrade >

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624398#comment-16624398 ] Hadoop QA commented on YARN-8804: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Resolved] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang resolved YARN-7599. Resolution: Fixed > [GPG] ApplicationCleaner in Global Policy Generator >

[jira] [Commented] (YARN-6456) Allow administrators to set a single ContainerRuntime for all containers

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624355#comment-16624355 ] Hadoop QA commented on YARN-6456: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-21 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624388#comment-16624388 ] Botong Huang commented on YARN-7599: Committed to YARN-7402. Thanks [~bibinchundatt] for the review!

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624395#comment-16624395 ] Hadoop QA commented on YARN-8804: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8696) [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624468#comment-16624468 ] Hadoop QA commented on YARN-8696: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624367#comment-16624367 ] Tao Yang commented on YARN-8774: Thanks [~eepayne] for the review. Attached v2 patch to improve the UT

[jira] [Commented] (YARN-8734) Readiness check for remote service

2018-09-21 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624346#comment-16624346 ] Gour Saha commented on YARN-8734: - Yup, it would be great to get [~billie.rinaldi] thoughts on the naming

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624382#comment-16624382 ] Hadoop QA commented on YARN-8774: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624440#comment-16624440 ] Hadoop QA commented on YARN-8774: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624473#comment-16624473 ] Hadoop QA commented on YARN-8665: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-21 Thread Giovanni Matteo Fumarola (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623929#comment-16623929 ] Giovanni Matteo Fumarola commented on YARN-8658: Pushed to Branch-2.  Thanks [~youchen] 

[jira] [Updated] (YARN-8696) [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async

2018-09-21 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-8696: --- Attachment: YARN-8696-branch-2.v6.patch > [AMRMProxy] FederationInterceptor upgrade: home sub-cluster

[jira] [Created] (YARN-8813) Improve debug messages for NM preemption of OPPORTUNISTIC containers

2018-09-21 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-8813: Summary: Improve debug messages for NM preemption of OPPORTUNISTIC containers Key: YARN-8813 URL: https://issues.apache.org/jira/browse/YARN-8813 Project: Hadoop YARN

[jira] [Commented] (YARN-8777) Container Executor C binary change to execute interactive docker command

2018-09-21 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623951#comment-16623951 ] Eric Badger commented on YARN-8777: --- I'm +1 (non-binding) on patch 003 > Container Executor C binary

[jira] [Updated] (YARN-8813) Improve debug messages for NM preemption of OPPORTUNISTIC containers

2018-09-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8813: - Attachment: YARN-8813-YARN-1011.00.patch > Improve debug messages for NM preemption of OPPORTUNISTIC

[jira] [Commented] (YARN-8813) Improve debug messages for NM preemption of OPPORTUNISTIC containers

2018-09-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623959#comment-16623959 ] Haibo Chen commented on YARN-8813: -- The patch adds a few debug log statements, hence no test is modified

[jira] [Commented] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-09-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623970#comment-16623970 ] Wangda Tan commented on YARN-8563: -- Thanks [~liuxun323],  The primary purpose of this ticket is to avoid

[jira] [Commented] (YARN-8811) Support Container Storage Interface (CSI) in YARN

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623889#comment-16623889 ] Eric Yang commented on YARN-8811: - [~cheersyang] Thank you for the proposal. There are several common

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623894#comment-16623894 ] Eric Yang commented on YARN-8785: - [~simonprewo] Can you post the patch 417.patch as YARN-8785.001.patch

[jira] [Commented] (YARN-8811) Support Container Storage Interface (CSI) in YARN

2018-09-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623901#comment-16623901 ] Wangda Tan commented on YARN-8811: -- Thanks [~cheersyang], [~sunilg] for the design doc.  In general the

[jira] [Commented] (YARN-8777) Container Executor C binary change to execute interactive docker command

2018-09-21 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623900#comment-16623900 ] Eric Yang commented on YARN-8777: - [~ebadger] Sounds good to me. [~Zian Chen] [~ebadger]

  1   2   >