[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997603#comment-14997603 ] Aashreya Ravi Shankar commented on MYRIAD-162: -- Looks like we are calculating the executor resources for mem and cpu twice in TaskFactory. I see more resources being consumed by each NM task. Is it possible to fix it for 0.1 release? Or should we address this as a part of the next release ? > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992220#comment-14992220 ] Santosh Marella commented on MYRIAD-162: PR https://github.com/apache/incubator-myriad/pull/32 has been merged. [~darinj] Do you want to open a new JIRA to address ports and close this one out? > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980728#comment-14980728 ] DarinJ commented on MYRIAD-162: --- I've got a pretty good handle on how to fix should be ready for review by Monday/Tuesday. > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus(*): 1.2; mem(*): > 1305.6; ports(*): [31005-31005,31006-31006,...] greater than offered > cpu(*):1, mem(*): 1400, ports(*): [ ... ], cpu(roleA): 3, mem(roleA): 1, > ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980815#comment-14980815 ] DarinJ commented on MYRIAD-162: --- Close but not quite. We want Myriad to be able to use Resources from both the frameworkUserRole AND the defaultRole (usually *). So we need to be aware of both, otherwise you're essentially statically partitioning your data center instead of reserving resources. The idea isn't hard and shouldn't require a major refactor. I'm prototyping now, will know more soon. If you wish to recreate simply add a role with some cpu resources to you're cluster and set frameworkRole to that role. You may need to ensure you're tasks require enough cpus/mem that they have to use the new role. > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980766#comment-14980766 ] Santosh Marella commented on MYRIAD-162: Looks like an important issue. Is the fix to add a "role" in the myriad-config-default.yml and have Myriad use that role when picking offers from Mesos? > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981545#comment-14981545 ] Santosh Marella commented on MYRIAD-162: Sounds good. I'm trying to get everyone close-in on the fixes/changes required for the 0.1.0 release. Without a fix for this, Myriad would still work when "roles" are not used in Mesos. Do you think the current behavior is good enough for a 0.1.0 release? > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-162) Myriad Not Correctly Dealing with Resources from Multiple Roles
[ https://issues.apache.org/jira/browse/MYRIAD-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981698#comment-14981698 ] DarinJ commented on MYRIAD-162: --- Without a fix Myriad will work if frameworkRole is not present or set to *, but fail badly if a frameworkRole is present. For a 0.1.0 release, the easiest nice things to do are 1. add a check if frameworkRole is present and log a warning and exit or 2. log and then set role to * anyway. After those options it's simply to fix it. I'm close to a fix (mem/cpu done working ports now) and would like to have it for 0.1.0, but it can wait for 0.1.1. > Myriad Not Correctly Dealing with Resources from Multiple Roles > --- > > Key: MYRIAD-162 > URL: https://issues.apache.org/jira/browse/MYRIAD-162 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.1.0 > Environment: Any where frameworkRole is not * >Reporter: DarinJ >Assignee: DarinJ > Fix For: Myriad 0.1.0 > > > When using Offers that have Resources from multiple roles, one needs to use > the setRole(String role) method to specify which role the resource belongs > to. Myriad currently doesn't do this which causes TASK_LOST, with an error > in the mesos-master log stating in "attempted to use cpus( * ): 1.2; mem( * > ): 1305.6; ports( * ): [31005-31005,31006-31006,...] greater than offered > cpu( * ):1, mem( * ): 1400, ports( * ): [ ... ], cpu(roleA): 3, mem(roleA): > 1, ports(roleA): [...]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
