[jira] [Updated] (YARN-10015) Correct the sample command in SLS README file

2020-01-28 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-10015: Component/s: yarn > Correct the sample command in SLS README file >

[jira] [Commented] (YARN-10015) Correct the sample command in SLS README file

2020-01-28 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025527#comment-17025527 ] Yufei Gu commented on YARN-10015: - Committed to trunk. Thanks for the patch, [~aihuaxu]. Thanks for the

[jira] [Updated] (YARN-10015) Correct the sample command in SLS README file

2020-01-28 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-10015: Fix Version/s: 3.3.0 > Correct the sample command in SLS README file >

[jira] [Commented] (YARN-10015) Correct the sample command in SLS README file

2020-01-27 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024762#comment-17024762 ] Yufei Gu commented on YARN-10015: - +1, will commit later. > Correct the sample command in SLS README

[jira] [Updated] (YARN-9537) Add configuration to disable AM preemption

2019-11-12 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9537: --- Fix Version/s: 3.3.0 Hadoop Flags: Reviewed > Add configuration to disable AM preemption >

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-11-12 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972654#comment-16972654 ] Yufei Gu commented on YARN-9537: Committed to trunk. Thanks for the contribution, [~cane]. Thanks for the

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-11-11 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972075#comment-16972075 ] Yufei Gu commented on YARN-9537: [~cane], Thanks for the patch. +1 for the patch 006. Will commit later.

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-11-11 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971376#comment-16971376 ] Yufei Gu commented on YARN-9537: Agreed with [~snemeth]. The production code shouldn't do the null

[jira] [Comment Edited] (YARN-9537) Add configuration to disable AM preemption

2019-11-07 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969450#comment-16969450 ] Yufei Gu edited comment on YARN-9537 at 11/7/19 5:41 PM: - Hi [~cane], sorry to

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-11-07 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969450#comment-16969450 ] Yufei Gu commented on YARN-9537: Hi [~cane], sorry to come late. Patch 003 looks good to me overall. Just

[jira] [Commented] (YARN-9940) avoid continuous scheduling thread crashes while sorting nodes get 'Comparison method violates its general contract'

2019-11-05 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967729#comment-16967729 ] Yufei Gu commented on YARN-9940: Hi [~kailiu_dev], added you to the contributor role, and assign this to

[jira] [Assigned] (YARN-9940) avoid continuous scheduling thread crashes while sorting nodes get 'Comparison method violates its general contract'

2019-11-05 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-9940: -- Assignee: kailiu_dev > avoid continuous scheduling thread crashes while sorting nodes get >

[jira] [Commented] (YARN-5913) Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext

2019-08-27 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917399#comment-16917399 ] Yufei Gu commented on YARN-5913: [~ykabusalah] feel free to take any Jira without assignee. > Consolidate

[jira] [Commented] (YARN-6425) Move out FS state dump code out of method update()

2019-08-27 Thread Yufei Gu (Jira)
[ https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917398#comment-16917398 ] Yufei Gu commented on YARN-6425: [~ykabusalah] feel free to do that. > Move out FS state dump code out of

[jira] [Commented] (YARN-2497) Fair scheduler should support strict node labels

2019-07-22 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890573#comment-16890573 ] Yufei Gu commented on YARN-2497: Hi [~chenzhaohang], AFAIK, FS doesn't support node label in any version.

[jira] [Assigned] (YARN-9537) Add configuration to disable AM preemption

2019-07-22 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-9537: -- Assignee: zhoukang > Add configuration to disable AM preemption >

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-07-22 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890572#comment-16890572 ] Yufei Gu commented on YARN-9537: Hi [~cane], added you to contributor, and assign this to you. Will you

[jira] [Updated] (YARN-9537) Add configuration to disable AM preemption

2019-07-18 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9537: --- Fix Version/s: (was: 3.1.2) (was: 3.2.0) > Add configuration to disable AM

[jira] [Commented] (YARN-9537) Add configuration to disable AM preemption

2019-06-17 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865860#comment-16865860 ] Yufei Gu commented on YARN-9537: Hi [~cane], thanks for the patch. Could you elaborate your use case? >

[jira] [Commented] (YARN-9537) Add configuration to support AM preemption

2019-05-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836625#comment-16836625 ] Yufei Gu commented on YARN-9537: FairScheduler doesn't prevent you from preempting the AM container. It

[jira] [Commented] (YARN-9520) fair scheduler: inter-queue-preemption.enabled, intra-queue-preemption.enabled options

2019-04-30 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830769#comment-16830769 ] Yufei Gu commented on YARN-9520: * inter-queue preemption will not happen among the applications of inside

[jira] [Commented] (YARN-9520) fair scheduler: inter-queue-preemption.enabled, intra-queue-preemption.enabled options

2019-04-30 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829993#comment-16829993 ] Yufei Gu commented on YARN-9520: Seems like you don't need queue A be fair policy. Why not set it to fifo

[jira] [Commented] (YARN-9520) fair scheduler: inter-queue-preemption.enabled, intra-queue-preemption.enabled options

2019-04-29 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829685#comment-16829685 ] Yufei Gu commented on YARN-9520: Could you elaborate the user case? > fair scheduler:

[jira] [Commented] (YARN-9214) Add AbstractYarnScheduler#getValidQueues method to remove duplication

2019-04-01 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807378#comment-16807378 ] Yufei Gu commented on YARN-9214: Committed to trunk. Thanks [~jiwq] for the contribution. Thanks

[jira] [Updated] (YARN-9214) Add AbstractYarnScheduler#getValidQueues method to remove duplication

2019-04-01 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9214: --- Summary: Add AbstractYarnScheduler#getValidQueues method to remove duplication (was: Add

[jira] [Commented] (YARN-9214) Add AbstractYarnScheduler#getValidQueues method to resolve duplicate code

2019-04-01 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807374#comment-16807374 ] Yufei Gu commented on YARN-9214: +1 > Add AbstractYarnScheduler#getValidQueues method to resolve

[jira] [Comment Edited] (YARN-9401) Fix `yarn version` print the version info is the same as `hadoop version`

2019-04-01 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807053#comment-16807053 ] Yufei Gu edited comment on YARN-9401 at 4/1/19 6:20 PM: Do we plan to release YARN

[jira] [Commented] (YARN-9401) Fix `yarn version` print the version info is the same as `hadoop version`

2019-04-01 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807053#comment-16807053 ] Yufei Gu commented on YARN-9401: Do we plan to release YARN separately? Probably never. With that, I

[jira] [Commented] (YARN-9401) Fix `yarn version` print the version info is the same as `hadoop version`

2019-03-31 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806357#comment-16806357 ] Yufei Gu commented on YARN-9401: Thank [~jiwq] for working on this. Looking more deeply, the hdfs command

[jira] [Commented] (YARN-9214) Add AbstractYarnScheduler#getValidQueues method to resolve duplicate code

2019-03-31 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806325#comment-16806325 ] Yufei Gu commented on YARN-9214: Thank [~jiwq] for working on this. {code} LOG.warn(errMsg);

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-25 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801412#comment-16801412 ] Yufei Gu commented on YARN-8967: Committed to trunk. Thanks [~wilfreds] for the contribution. > Change

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-23 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799736#comment-16799736 ] Yufei Gu commented on YARN-8967: +1. Will commit later. > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-22 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798814#comment-16798814 ] Yufei Gu commented on YARN-8967: Hi [~wilfreds], thanks for the patch. 3) Yeah, the xml DOM looks like a

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-20 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796846#comment-16796846 ] Yufei Gu commented on YARN-8967: Hi [~wilfreds], the patch v9 looks really good. {quote} Based on all

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-13 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791411#comment-16791411 ] Yufei Gu commented on YARN-8967: Hi [~wilfreds], thanks for the patch. Some comments: 1. Nice cleanup in

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9298: --- Fix Version/s: 3.3.0 > Implement FS placement rules using PlacementRule interface >

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784193#comment-16784193 ] Yufei Gu commented on YARN-9298: Committed to trunk. Thanks [~wilfreds] for the contribution. > Implement

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783618#comment-16783618 ] Yufei Gu commented on YARN-9298: +1 for the patch v6. Will commit later. > Implement FS placement rules

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782911#comment-16782911 ] Yufei Gu commented on YARN-9298: [~wilfreds], thanks for the patch. Looks really good! Just some nits: 1.

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-28 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780302#comment-16780302 ] Yufei Gu commented on YARN-9298: Hi [~wilfreds], thanks for the patch. I am glad we are moving forward.

[jira] [Comment Edited] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776175#comment-16776175 ] Yufei Gu edited comment on YARN-9298 at 2/24/19 9:31 AM: - Hi [~wilfreds], thanks

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-02-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776184#comment-16776184 ] Yufei Gu commented on YARN-9278: bq. If our cluster has a lot of long-running jobs, the above method is

[jira] [Comment Edited] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776175#comment-16776175 ] Yufei Gu edited comment on YARN-9298 at 2/24/19 9:19 AM: - Hi [~wilfreds], thanks

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776175#comment-16776175 ] Yufei Gu commented on YARN-9298: Hi [~wilfreds], thanks for the patch. It is really nice to add these unit

[jira] [Comment Edited] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-02-20 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773299#comment-16773299 ] Yufei Gu edited comment on YARN-9278 at 2/20/19 7:47 PM: - Hi [~uranus], this seems

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-02-20 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773299#comment-16773299 ] Yufei Gu commented on YARN-9278: Hi [~uranus], this seems a perf issue for a busy large cluster due to the

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-16 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770247#comment-16770247 ] Yufei Gu commented on YARN-9298: Hi [~wilfreds], thanks for splitting and provide the patch. Some

[jira] [Commented] (YARN-9277) Add more restrictions In FairScheduler Preemption

2019-02-12 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766287#comment-16766287 ] Yufei Gu commented on YARN-9277: Hi [~uranus], some general comments, I haven't looked at the code yet.

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759562#comment-16759562 ] Yufei Gu commented on YARN-8967: Hi [~wilfreds], the patch v4 doesn't apply to the trunk. Can you rebase

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-8967: --- Fix Version/s: (was: 3.3) > Change FairScheduler to use PlacementRule interface >

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-8967: --- Fix Version/s: 3.3 > Change FairScheduler to use PlacementRule interface >

[jira] [Commented] (YARN-9041) Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt

2018-12-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16709067#comment-16709067 ] Yufei Gu commented on YARN-9041: Committed to trunk. Thanks [~jiwq] for working on this. Thanks [~Steven

[jira] [Updated] (YARN-9041) Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt

2018-12-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9041: --- Fix Version/s: 3.2.1 > Performance Optimization of method > FSPreemptionThread#identifyContainersToPreempt >

[jira] [Updated] (YARN-9041) Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt

2018-12-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9041: --- Summary: Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt (was: Performance

[jira] [Updated] (YARN-9041) Performance Optimization of FSPreemptionThread#identifyContainersToPreempt method

2018-12-04 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9041: --- Summary: Performance Optimization of FSPreemptionThread#identifyContainersToPreempt method (was: Optimize

[jira] [Updated] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-12-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9041: --- Affects Version/s: 3.1.1 > Optimize FSPreemptionThread#identifyContainersToPreempt method >

[jira] [Commented] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-12-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708242#comment-16708242 ] Yufei Gu commented on YARN-9041: The last patch looks good. +1 for the patch v7. Will commit this soon. >

[jira] [Updated] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-12-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9041: --- Component/s: fairscheduler > Optimize FSPreemptionThread#identifyContainersToPreempt method >

[jira] [Commented] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-12-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707857#comment-16707857 ] Yufei Gu commented on YARN-9041: Thanks for the patch. Some nits: # {{ * @return list preemptable

[jira] [Commented] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-11-29 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704014#comment-16704014 ] Yufei Gu commented on YARN-9041: Hi [~jiwq], the patch v5 looks good in terms of logic. Some nits: # Can

[jira] [Comment Edited] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-11-29 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704014#comment-16704014 ] Yufei Gu edited comment on YARN-9041 at 11/29/18 11:21 PM: --- Hi [~jiwq], the

[jira] [Commented] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-11-28 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702308#comment-16702308 ] Yufei Gu commented on YARN-9041: There is an error in test build which isn't related to your patch. I'll

[jira] [Commented] (YARN-9041) Optimize FSPreemptionThread#identifyContainersToPreempt method

2018-11-26 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699511#comment-16699511 ] Yufei Gu commented on YARN-9041: Hi [~jiwq], thanks for the patch. I like the idea to shrink the search

[jira] [Commented] (YARN-9005) FairScheduler maybe preempt the AM container

2018-11-16 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690178#comment-16690178 ] Yufei Gu commented on YARN-9005: Hi [~jiwq], sorry for the late response. Please clarify what performance

[jira] [Commented] (YARN-9005) FairScheduler maybe preempt the AM container

2018-11-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682243#comment-16682243 ] Yufei Gu commented on YARN-9005: It is by design that AM containers can be preempted. YARN-5830 did the

[jira] [Updated] (YARN-9005) FairScheduler maybe preempt the AM container

2018-11-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-9005: --- Component/s: scheduler preemption fairscheduler > FairScheduler maybe preempt the AM

[jira] [Commented] (YARN-8978) For fair scheduler, application with higher priority should also get priority resources for running AM

2018-11-07 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678577#comment-16678577 ] Yufei Gu commented on YARN-8978: [~qiuliang988], not sure if you still need this jira but you shouldn't

[jira] [Commented] (YARN-8969) Change the return type to generic type of AbstractYarnScheduler#getNodeTracker

2018-11-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675958#comment-16675958 ] Yufei Gu commented on YARN-8969: [~eepayne], it is probably fine in this case. {{AbstractYarnScheduler}}

[jira] [Commented] (YARN-8792) Revisit FairScheduler QueuePlacementPolicy

2018-09-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631207#comment-16631207 ] Yufei Gu commented on YARN-8792: [~HCOONa], I've added you as a contributor, so that you can assign these

[jira] [Commented] (YARN-7794) SLSRunner is not loading timeline service jars causing failure

2018-09-06 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606480#comment-16606480 ] Yufei Gu commented on YARN-7794: [~jhung], the patch looks good to me. > SLSRunner is not loading

[jira] [Commented] (YARN-8632) Threads in SLS quit without logging exception

2018-08-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592032#comment-16592032 ] Yufei Gu commented on YARN-8632: +1 for the patch v4. Committed to trunk. Thanks [~luxianghao] for the

[jira] [Updated] (YARN-8632) Threads in SLS quit without logging exception

2018-08-24 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-8632: --- Summary: Threads in SLS quit without logging exception (was: No data in file realtimetrack.json after

[jira] [Commented] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-23 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590570#comment-16590570 ] Yufei Gu commented on YARN-8632: [~luxianghao], thanks for the patch. Nice finding. +1 for the patch v3.

[jira] [Commented] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-20 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586344#comment-16586344 ] Yufei Gu commented on YARN-8632: For that sake, we need to "setUncaughtExceptionHandler" for the thread,

[jira] [Commented] (YARN-5139) [Umbrella] Move YARN scheduler towards global scheduler

2018-08-16 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583318#comment-16583318 ] Yufei Gu commented on YARN-5139: [~zhuqi], done. > [Umbrella] Move YARN scheduler towards global

[jira] [Assigned] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe

2018-08-14 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-8655: -- Assignee: Zhaohui Xin > FairScheduler: FSStarvedApps is not thread safe >

[jira] [Commented] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe

2018-08-14 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580139#comment-16580139 ] Yufei Gu commented on YARN-8655: Hi [~uranus], added you to the contributor list and assigned this to

[jira] [Commented] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-13 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16578945#comment-16578945 ] Yufei Gu commented on YARN-8632: Thanks for the patch. Some comments: # It is not a good practice to catch

[jira] [Comment Edited] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-10 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576875#comment-16576875 ] Yufei Gu edited comment on YARN-8632 at 8/10/18 9:28 PM: - Your patch doesn't apply

[jira] [Commented] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-10 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576875#comment-16576875 ] Yufei Gu commented on YARN-8632: Your patch doesn't apply to trunk? You said the bug is in trunk as well,

[jira] [Assigned] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-10 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-8632: -- Assignee: Xianghao Lu > No data in file realtimetrack.json after running SchedulerLoadSimulator >

[jira] [Commented] (YARN-8632) No data in file realtimetrack.json after running SchedulerLoadSimulator

2018-08-10 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1657#comment-1657 ] Yufei Gu commented on YARN-8632: Added you to the contributor list and assign this to you. Will review

[jira] [Commented] (YARN-8639) Sort queue and apps in fair schduler using a separate thread

2018-08-10 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575926#comment-16575926 ] Yufei Gu commented on YARN-8639: Close it or take this chance to do a little bit test. Either way works

[jira] [Commented] (YARN-8639) Sort queue and apps in fair schduler using a separate thread

2018-08-09 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575302#comment-16575302 ] Yufei Gu commented on YARN-8639: We need to quantify a little bit before we'd make any non-trivial change.

[jira] [Commented] (YARN-6636) Fair Scheduler: respect node labels at resource request level

2018-08-07 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572303#comment-16572303 ] Yufei Gu commented on YARN-6636: There are multiple ways to approach node labeling in Fair Scheduler. The

[jira] [Commented] (YARN-8495) Priority scheduling support in FairShare scheduler

2018-07-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533902#comment-16533902 ] Yufei Gu commented on YARN-8495: Thanks [~Dillon.] for filing this. The implementation conflicts with the

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-06-28 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526924#comment-16526924 ] Yufei Gu commented on YARN-8468: Sounds good to me. Thanks [~mrbillau]. > Limit container sizes per queue

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-06-28 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526869#comment-16526869 ] Yufei Gu commented on YARN-8468: [~bsteinbach] since you filed this jira and provided the patch, you have

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-06-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525773#comment-16525773 ] Yufei Gu commented on YARN-8468: It seems a benign feature in terms of how it impact the existing

[jira] [Commented] (YARN-8184) Too many metrics if containerLocalizer/ResourceLocalizationService uses ReadWriteDiskValidator

2018-06-22 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520773#comment-16520773 ] Yufei Gu commented on YARN-8184: Committed to trunk. Thanks for the review, [~haibochen]. > Too many

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-13 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511503#comment-16511503 ] Yufei Gu commented on YARN-8394: Hi [~cheersyang], Let me clarify a little bit. The code logic should be:

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-12 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510625#comment-16510625 ] Yufei Gu commented on YARN-8394: LGTM, can you file a jira for the code change? > Improve data locality

[jira] [Commented] (YARN-8406) Do the improvement to the FSLeafQueue about calculating fair share for apps

2018-06-08 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506571#comment-16506571 ] Yufei Gu commented on YARN-8406: You can probably move your patch and test result to YARN-7467 since I've

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-08 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505822#comment-16505822 ] Yufei Gu commented on YARN-8394: Sounds good to me. > Improve data locality documentation for Capacity

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-07 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505726#comment-16505726 ] Yufei Gu commented on YARN-8394: bq. This can be done by setting

[jira] [Commented] (YARN-8406) Do the improvement to the FSLeafQueue about calculating fair share for apps

2018-06-07 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504915#comment-16504915 ] Yufei Gu commented on YARN-8406: [~zhuqi], there is no need to create this subtask. You can ask

[jira] [Resolved] (YARN-8406) Do the improvement to the FSLeafQueue about calculating fair share for apps

2018-06-07 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu resolved YARN-8406. Resolution: Duplicate > Do the improvement to the FSLeafQueue about calculating fair share for apps >

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-06 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503552#comment-16503552 ] Yufei Gu commented on YARN-8394: Make senses to me assuming that the Cloud solution still uses CS/FS as

[jira] [Commented] (YARN-8394) Improve data locality documentation for Capacity Scheduler

2018-06-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502479#comment-16502479 ] Yufei Gu commented on YARN-8394: Hi [~cheersyang], thanks for filing this. Can you elaborate on this? bq.

[jira] [Commented] (YARN-5139) [Umbrella] Move YARN scheduler towards global scheduler

2018-06-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502076#comment-16502076 ] Yufei Gu commented on YARN-5139: [~zhuqi], you are welcome to contribute to Fair Scheduler. It's not a

  1   2   3   4   5   6   7   8   9   10   >