[jira] [Commented] (YARN-1187) Add discrete event-based simulation to yarn scheduler simulator

2021-01-07 Thread Wei Yan (Jira)
[ https://issues.apache.org/jira/browse/YARN-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260888#comment-17260888 ] Wei Yan commented on YARN-1187: --- [~afchung90] feel free to take it :) > Add discrete event-based simulation

[jira] [Commented] (YARN-8135) Hadoop {Submarine} Project: Simple and scalable deployment of deep learning training / serving jobs on Hadoop

2018-04-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432502#comment-16432502 ] Wei Yan commented on YARN-8135: --- {quote}Think this should be renamed to YARN-Submarine though. I'm not sure

[jira] [Commented] (YARN-4488) CapacityScheduler: Compute per-container allocation latency and roll up to get per-application and per-queue

2018-03-08 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391597#comment-16391597 ] Wei Yan commented on YARN-4488: --- Thanks for pinging, [~leftnoteasy]. I created YARN-7844 previously, which

[jira] [Commented] (YARN-7844) Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX

2018-01-30 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345657#comment-16345657 ] Wei Yan commented on YARN-7844: --- Put a new diff 001.patch to fix some checkstyles and findbugs issues.

[jira] [Updated] (YARN-7844) Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX

2018-01-30 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-7844: -- Attachment: YARN-7844.001.patch > Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX >

[jira] [Commented] (YARN-7844) Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX

2018-01-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344581#comment-16344581 ] Wei Yan commented on YARN-7844: --- After enabling the metrics, the JMX will have one more item like:

[jira] [Updated] (YARN-7844) Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX

2018-01-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-7844: -- Attachment: YARN-7844.000.patch > Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX >

[jira] [Created] (YARN-7844) Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX

2018-01-29 Thread Wei Yan (JIRA)
Wei Yan created YARN-7844: - Summary: Expose metrics for scheduler operation (allocate, schedulerEvent) to JMX Key: YARN-7844 URL: https://issues.apache.org/jira/browse/YARN-7844 Project: Hadoop YARN

[jira] [Commented] (YARN-7672) hadoop-sls can not simulate huge scale of YARN

2017-12-20 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298689#comment-16298689 ] Wei Yan commented on YARN-7672: --- bq. All of NM and AM simulators are all cpu type of task. So cpu.load will

[jira] [Commented] (YARN-7672) hadoop-sls can not simulate huge scale of YARN

2017-12-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296427#comment-16296427 ] Wei Yan commented on YARN-7672: --- Interesting to know you guys running 10K node cluster. Do you have more info

[jira] [Updated] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-11-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-6851: -- Fix Version/s: 2.9.1 2.10.0 3.1.0 > Capacity Scheduler: document configs

[jira] [Commented] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-11-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271703#comment-16271703 ] Wei Yan commented on YARN-6851: --- Thanks for the review, [~wangda]. Committed to branch-2, branch-2.9 and

[jira] [Updated] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-11-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-6851: -- Attachment: YARN-6851-branch-2.001.patch Also trigger branch-2 Jenkins > Capacity Scheduler: document configs

[jira] [Commented] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-08-14 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126015#comment-16126015 ] Wei Yan commented on YARN-6851: --- [~sunilg], [~wangda], any comments? > Capacity Scheduler: document configs

[jira] [Commented] (YARN-2098) App priority support in Fair Scheduler

2017-08-11 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124307#comment-16124307 ] Wei Yan commented on YARN-2098: --- Thanks for the reminder, [~sunilg]. Sorry I forgot some old jiras. To align

[jira] [Updated] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-08-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-6851: -- Attachment: YARN-6851.001.patch > Capacity Scheduler: document configs for controlling # containers allowed to

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-08-03 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113123#comment-16113123 ] Wei Yan commented on YARN-4161: --- Yes, here we introduce one more condition check. But by default, the new

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-08-01 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109725#comment-16109725 ] Wei Yan commented on YARN-4161: --- [~sunilg] could u help review the latest patch? :) thanks. > Capacity

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-26 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102021#comment-16102021 ] Wei Yan commented on YARN-4161: --- The TestDelegationTokenRenewer test error looks un-related. [~sunilg], any

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-25 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.006.patch Good catch, [~sunilg]. Update a new patch to include that check. > Capacity

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-20 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095165#comment-16095165 ] Wei Yan commented on YARN-4161: --- The failed testcase passed locally, looks like not related to this patch.

[jira] [Created] (YARN-6851) Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

2017-07-20 Thread Wei Yan (JIRA)
Wei Yan created YARN-6851: - Summary: Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat Key: YARN-6851 URL: https://issues.apache.org/jira/browse/YARN-6851

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.005.patch > Capacity Scheduler : Assign single or multiple containers per heart beat >

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.004.patch Update patch to fix checkstyle issues. The test errors look not related to this

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-07-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.003.patch rebase the patch > Capacity Scheduler : Assign single or multiple containers

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-06-30 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.002.patch A new diff based on previous comments: (1) Rename the config fields to follow CS

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-06-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068542#comment-16068542 ] Wei Yan commented on YARN-4161: --- {quote} I got your point. I was trying to point out to an existing

[jira] [Commented] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-06-28 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067299#comment-16067299 ] Wei Yan commented on YARN-4161: --- Thanks for the comments [~sunilg]. For the configuration names, I followed

[jira] [Updated] (YARN-4161) Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

2017-06-27 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-4161: -- Attachment: YARN-4161.patch.1 Rebase for trunk, by moving the assignMultiple check to scheduler-level, instead

[jira] [Commented] (YARN-2098) App priority support in Fair Scheduler

2016-08-28 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444764#comment-15444764 ] Wei Yan commented on YARN-2098: --- Looks like the old patch cannot be rebased. I'll code a new one. > App

[jira] [Commented] (YARN-2098) App priority support in Fair Scheduler

2016-08-09 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414756#comment-15414756 ] Wei Yan commented on YARN-2098: --- Sorry... missed this ticket... I'll grab some time to fix it. > App

[jira] [Commented] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-21 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388029#comment-15388029 ] Wei Yan commented on YARN-1126: --- Thanks, [~rohithsharma]. > Add validation of users input nodes-states

[jira] [Commented] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-20 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386233#comment-15386233 ] Wei Yan commented on YARN-1126: --- Not sure about the test case failures. Tried locally, and it was ok.

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-18 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: YARN-1126-005.patch Good catch, [~rohithsharma]. Update a new patch to add the missing tests. > Add

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-15 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: YARN-1126-004.patch trigger the jenkins. > Add validation of users input nodes-states options to

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-15 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: (was: YARN-1126-004.patch) > Add validation of users input nodes-states options to nodes CLI >

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-07-15 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: YARN-1126-004.patch Rebasing the patch. > Add validation of users input nodes-states options to

[jira] [Commented] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2016-06-29 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355482#comment-15355482 ] Wei Yan commented on YARN-1126: --- Thanks, [~rohithsharma]. will do it soon. > Add validation of users input

[jira] [Commented] (YARN-5035) FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on

2016-05-03 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15269125#comment-15269125 ] Wei Yan commented on YARN-5035: --- But if continuous-scheduling is turned on, we can always set maxAssign=1 to

[jira] [Commented] (YARN-2098) App priority support in Fair Scheduler

2015-08-06 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660626#comment-14660626 ] Wei Yan commented on YARN-2098: --- [~sunilg], [~ashwinshankar77]. Sure, will rebase a new one.

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2015-07-08 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619317#comment-14619317 ] Wei Yan commented on YARN-2194: --- [~vinodkv], Thanks for pointing it out. IMO, I don't think

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2015-07-01 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610475#comment-14610475 ] Wei Yan commented on YARN-2194: --- Thanks, [~vvasudev]. Cgroups cease to work in RHEL7

[jira] [Updated] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-18 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2194: -- Attachment: YARN-2194-5.patch Updated a new patch addressing [~sidharta-s] comments. I put the '%' constant to

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-18 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592125#comment-14592125 ] Wei Yan commented on YARN-2194: --- [~sidharta-s], thanks for pointing out. Will fix it soon.

[jira] [Updated] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-18 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2194: -- Attachment: YARN-2194-6.patch Cgroups cease to work in RHEL7 --

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14583404#comment-14583404 ] Wei Yan commented on YARN-2194: --- [~kasha], [~sidharta-s], thanks for the comments. Looking

[jira] [Updated] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2194: -- Attachment: YARN-2194-4.patch Uploaded a patch by replacing comma with '%'. Cgroups cease to work in RHEL7

[jira] [Commented] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-08 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577496#comment-14577496 ] Wei Yan commented on YARN-2194: --- [~sidharta-s], yes, working on it. Cgroups cease to work

[jira] [Updated] (YARN-2194) Cgroups cease to work in RHEL7

2015-06-01 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2194: -- Attachment: YARN-2194-3.patch Thanks, [~kasha]. Updated a patch adding more comments. Cgroups cease to work in

[jira] [Updated] (YARN-2194) Add Cgroup support for RedHat 7

2015-05-31 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2194: -- Attachment: YARN-2194-2.patch In Redhat 7, the controller for cpu is /sys/fs/cgroup/cpu,cpuacct. However, this

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2015-05-23 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: YARN-1126-003.patch Add validation of users input nodes-states options to nodes CLI

[jira] [Updated] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2015-05-23 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1126: -- Attachment: YARN-1126-002.patch Rebase the patch Add validation of users input nodes-states options to nodes

[jira] [Commented] (YARN-1126) Add validation of users input nodes-states options to nodes CLI

2015-05-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14556409#comment-14556409 ] Wei Yan commented on YARN-1126: --- [~djp], will update a new patch soon. Add validation of

[jira] [Commented] (YARN-2151) FairScheduler option for global preemption within hierarchical queues

2015-05-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537557#comment-14537557 ] Wei Yan commented on YARN-2151: --- [~kasha], yes, we have coupled of preemption-related patched

[jira] [Commented] (YARN-2194) Add Cgroup support for RedHat 7

2015-05-07 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533256#comment-14533256 ] Wei Yan commented on YARN-2194: --- [~vinodkv], discussed about this with Karthik offline. We

[jira] [Commented] (YARN-2618) Avoid over-allocation of disk resources

2015-05-04 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526791#comment-14526791 ] Wei Yan commented on YARN-2618: --- Thanks, [~djp], I'll rebase the patch. Avoid

[jira] [Commented] (YARN-2619) NodeManager: Add cgroups support for disk I/O isolation

2015-04-27 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514527#comment-14514527 ] Wei Yan commented on YARN-2619: --- Thanks, [~vvasudev]. NodeManager: Add cgroups support for

[jira] [Commented] (YARN-1021) Yarn Scheduler Load Simulator

2015-04-24 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14511246#comment-14511246 ] Wei Yan commented on YARN-1021: --- [~harman], could u send me an email including your detail

[jira] [Commented] (YARN-3525) Rename fair scheduler properties increment-allocation-mb and increment-allocation-vcores

2015-04-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507381#comment-14507381 ] Wei Yan commented on YARN-3525: --- Hi, [~bibinchundatt], I think the reason is These properties

[jira] [Commented] (YARN-2618) Avoid over-allocation of disk resources

2015-04-13 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492748#comment-14492748 ] Wei Yan commented on YARN-2618: --- [~kasha], help commit the patch? Avoid over-allocation of

[jira] [Updated] (YARN-2618) Avoid over-allocation of disk resources

2015-04-06 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2618: -- Attachment: YARN-2618-7.patch Fix the testing errors. Avoid over-allocation of disk resources

[jira] [Updated] (YARN-2140) Add support for network IO isolation/scheduling for containers

2015-04-03 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2140: -- Assignee: Sidharta Seethana (was: Wei Yan) Add support for network IO isolation/scheduling for containers

[jira] [Commented] (YARN-2666) TestFairScheduler.testContinuousScheduling fails Intermittently

2015-03-31 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389105#comment-14389105 ] Wei Yan commented on YARN-2666: --- Hey, [~zxu], I'll take a look of your patch later today.

[jira] [Updated] (YARN-2666) TestFairScheduler.testContinuousScheduling fails Intermittently

2015-03-31 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2666: -- Assignee: zhihai xu (was: Wei Yan) TestFairScheduler.testContinuousScheduling fails Intermittently

[jira] [Commented] (YARN-2666) TestFairScheduler.testContinuousScheduling fails Intermittently

2015-03-31 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389344#comment-14389344 ] Wei Yan commented on YARN-2666: --- [~zxu], thanks for your patch. Good catch of the bug. The

[jira] [Updated] (YARN-2618) Avoid over-allocation of disk resources

2015-03-26 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2618: -- Attachment: YARN-2618-6.patch Rebase the patch. Avoid over-allocation of disk resources

[jira] [Commented] (YARN-2618) Avoid over-allocation of disk resources

2015-03-25 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380035#comment-14380035 ] Wei Yan commented on YARN-2618: --- [~kasha], sure, will do it soon. Avoid over-allocation of

[jira] [Commented] (YARN-3126) FairScheduler: queue's usedResource is always more than the maxResource limit

2015-02-06 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14309865#comment-14309865 ] Wei Yan commented on YARN-3126: --- [~Xia Hu], I checked the latest trunk version. The problem

[jira] [Commented] (YARN-3126) FairScheduler: queue's usedResource is always more than the maxResource limit

2015-02-04 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306630#comment-14306630 ] Wei Yan commented on YARN-3126: --- [~Xia Hu], does this problem still exist in trunk version?

[jira] [Updated] (YARN-1393) Add how-to-use instruction in README for Yarn Scheduler Load Simulator

2015-01-24 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1393: -- Attachment: YARN-1393-3.patch Add how-to-use instruction in README for Yarn Scheduler Load Simulator

[jira] [Commented] (YARN-2194) Add Cgroup support for RedHat 7

2015-01-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287645#comment-14287645 ] Wei Yan commented on YARN-2194: --- Sure, will update a new patch by combing [~bcwalrus]

[jira] [Commented] (YARN-2194) Add Cgroup support for RedHat 7

2015-01-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287644#comment-14287644 ] Wei Yan commented on YARN-2194: --- Sure, will update a new patch by combing [~bcwalrus]

[jira] [Updated] (YARN-1393) Add how-to-use instruction in README for Yarn Scheduler Load Simulator

2015-01-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-1393: -- Attachment: YARN-1393-2.patch Thanks, [~kasha], I updated a new patch by clearing the file path. Add

[jira] [Commented] (YARN-3020) n similar addContainerRequest()s produce n*(n+1)/2 containers

2015-01-21 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285984#comment-14285984 ] Wei Yan commented on YARN-3020: --- Hi, [~peterdkirchner], try to understand clearly your point.

[jira] [Commented] (YARN-1021) Yarn Scheduler Load Simulator

2015-01-20 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14284099#comment-14284099 ] Wei Yan commented on YARN-1021: --- Thanks for the reply, [~kasha]. [~fcolzada], sorry for the

[jira] [Commented] (YARN-3054) Preempt policy in FairScheduler may cause mapreduce job never finish

2015-01-14 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14277174#comment-14277174 ] Wei Yan commented on YARN-3054: --- Hi, [~peng.zhang]. Firstly, FairScheduler will check whether

[jira] [Commented] (YARN-2791) Add Disk as a resource for scheduling

2015-01-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274171#comment-14274171 ] Wei Yan commented on YARN-2791: --- [~srivas], YARN-2618 is a simple solution that limits the

[jira] [Commented] (YARN-810) Support CGroup ceiling enforcement on CPU

2015-01-09 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271339#comment-14271339 ] Wei Yan commented on YARN-810: -- The latest TestResourceLocalizationService failure is

[jira] [Commented] (YARN-3020) n similar addContainerRequest()s produce n*(n+1)/2 containers

2015-01-09 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271504#comment-14271504 ] Wei Yan commented on YARN-3020: --- [~peterdkirchner], I run the following test code in

[jira] [Updated] (YARN-810) Support CGroup ceiling enforcement on CPU

2015-01-08 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-810: - Attachment: YARN-810-6.patch Update a patch to fix the test failures. Support CGroup ceiling enforcement on CPU

[jira] [Commented] (YARN-1177) Support automatic failover using ZKFC

2015-01-05 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264789#comment-14264789 ] Wei Yan commented on YARN-1177: --- [~ajisakaa], feel free to grab. thanks. Support automatic

[jira] [Updated] (YARN-810) Support CGroup ceiling enforcement on CPU

2014-12-23 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-810: - Attachment: YARN-810-5.patch Update a new patch which encodes the per-app cpu_ceiling_enforce setting in

[jira] [Resolved] (YARN-2420) Fair Scheduler: dynamically update yarn.scheduler.fair.max.assign based on cluster load

2014-12-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan resolved YARN-2420. --- Resolution: Won't Fix Fair Scheduler: dynamically update yarn.scheduler.fair.max.assign based on cluster

[jira] [Commented] (YARN-2420) Fair Scheduler: dynamically update yarn.scheduler.fair.max.assign based on cluster load

2014-12-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256072#comment-14256072 ] Wei Yan commented on YARN-2420: --- sure, I'll close it. Fair Scheduler: dynamically update

[jira] [Updated] (YARN-810) Support CGroup ceiling enforcement on CPU

2014-12-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-810: - Attachment: YARN-810-3.patch Rebase a new patch for review. Support CGroup ceiling enforcement on CPU

[jira] [Updated] (YARN-810) Support CGroup ceiling enforcement on CPU

2014-12-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-810: - Attachment: YARN-810-4.patch Support CGroup ceiling enforcement on CPU -

[jira] [Commented] (YARN-2655) AllocatedGB/AvailableGB in nodemanager JMX showing only integer values

2014-12-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254202#comment-14254202 ] Wei Yan commented on YARN-2655: --- [~kasha], sure, will do it soon. AllocatedGB/AvailableGB

[jira] [Commented] (YARN-2655) AllocatedGB/AvailableGB in nodemanager JMX showing only integer values

2014-12-19 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254225#comment-14254225 ] Wei Yan commented on YARN-2655: --- Problem already solved in YARN-1156. Closing it.

[jira] [Commented] (YARN-2618) Avoid over-allocation of disk resources

2014-12-17 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250193#comment-14250193 ] Wei Yan commented on YARN-2618: --- Thanks, [~kasha] Avoid over-allocation of disk resources

[jira] [Updated] (YARN-2850) DistributedShell: Add support for disk I/O request

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2850: -- Attachment: YARN-2850-1.patch Attach the first patch for review, based on YARN-2618. DistributedShell: Add

[jira] [Updated] (YARN-2851) YarnClient: Add support for disk I/O resource/request information

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2851: -- Attachment: YARN-2851-1.patch Put an initial patch for review, based on YARN-2618. YarnClient: Add support for

[jira] [Updated] (YARN-2852) WebUI Metrics: Add disk I/O resource information to the web ui and metrics

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2852: -- Summary: WebUI Metrics: Add disk I/O resource information to the web ui and metrics (was: WebUI: Add disk I/O

[jira] [Updated] (YARN-2852) WebUI Metrics: Add disk I/O resource information to the web ui and metrics

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2852: -- Attachment: YARN-2852-1.patch Patch for review, based on YARN-2618. WebUI Metrics: Add disk I/O resource

[jira] [Updated] (YARN-2620) FairScheduler: Add disk I/O resource to the DRF implementation

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2620: -- Attachment: YARN-2620-1.patch FairScheduler: Add disk I/O resource to the DRF implementation

[jira] [Commented] (YARN-2620) FairScheduler: Add disk I/O resource to the DRF implementation

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244664#comment-14244664 ] Wei Yan commented on YARN-2620: --- Initial patch, based on YARN-2168. Some testcases fail,

[jira] [Commented] (YARN-2620) FairScheduler: Add disk I/O resource to the DRF implementation

2014-12-12 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244666#comment-14244666 ] Wei Yan commented on YARN-2620: --- typo... should be YARN-2618. FairScheduler: Add disk I/O

[jira] [Updated] (YARN-2618) Avoid over-allocation of disk resources

2014-12-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2618: -- Attachment: YARN-2618-4.patch Update a new patch fixing the test failures. Avoid over-allocation of disk

[jira] [Updated] (YARN-2618) Avoid over-allocation of disk resources

2014-12-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2618: -- Attachment: YARN-2618-5.patch Avoid over-allocation of disk resources ---

[jira] [Commented] (YARN-2618) Avoid over-allocation of disk resources

2014-12-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241486#comment-14241486 ] Wei Yan commented on YARN-2618: --- The four failed testcases passed locally. Some of the

[jira] [Updated] (YARN-2619) NodeManager: Add cgroups support for disk I/O isolation

2014-12-10 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated YARN-2619: -- Attachment: YARN-2619-1.patch Prelim patch for reviewing. The patch is based on YARN-2618. NodeManager: Add

  1   2   3   4   5   >