[jira] [Updated] (MESOS-5936) Operator SUBSCRIBE api should provdide more task metadata than just state changes
[ https://issues.apache.org/jira/browse/MESOS-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-5936: -- Target Version/s: 1.1.0 > Operator SUBSCRIBE api should provdide more task metadata than just state > changes > - > > Key: MESOS-5936 > URL: https://issues.apache.org/jira/browse/MESOS-5936 > Project: Mesos > Issue Type: Improvement > Components: HTTP API, json api >Affects Versions: 1.0.0 >Reporter: Steven Schlansker >Assignee: Zhitao Li > Labels: mesosphere > Fix For: 1.1.0 > > > I am starting to use the new Operator event subscription API to get notified > of task changes. The initial TASK_STAGING event has a good amount of > information, but unfortunately future events like TASK_RUNNING include almost > no metadata. This means that any task information that cannot be determined > until the task launches (in my particular case, the IP address assigned by > the Docker containerizer) is not available through the event stream. > Here is a gist of a single task that was launched, comparing the information > in 'state.json' vs the subscribed events: > https://gist.github.com/stevenschlansker/c1d32aa9ce37a73f9c4d64347397d3b8 > Note particularly how the IP address never shows in the event stream. > Task updates should expose the task information that changed. If this is too > onerous, maybe the subscription call can take optional sets of data to > include, with the first one being additional task metadata. > A possible workaround is to use the task events to fetch 'state.json' > separately, but this is inherently racy and totally undermines the utility of > the event stream api. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Ma updated MESOS-6247: Comment: was deleted (was: They can not share resources with each other in two roles.) > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547250#comment-15547250 ] Klaus Ma commented on MESOS-6247: - They can not share resources with each other in two roles. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547251#comment-15547251 ] Klaus Ma commented on MESOS-6247: - They can not share resources with each other in two roles. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547159#comment-15547159 ] Klaus Ma edited comment on MESOS-6247 at 10/5/16 12:27 AM: --- Yes, hierarchical role is the solution for this case; do you have target date for that feature? If not, set weight to framework is the fastest solution I can image. was (Author: klaus1982): Yes, hierarchical role is the target solution for this case; do you have target date for that feature? If not, set weight to framework is the fastest solution I can image. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547159#comment-15547159 ] Klaus Ma edited comment on MESOS-6247 at 10/5/16 12:27 AM: --- Yes, hierarchical role is the solution for this case; do you have target date for that feature? If not, set weight to framework is the solution I can image for now. was (Author: klaus1982): Yes, hierarchical role is the solution for this case; do you have target date for that feature? If not, set weight to framework is the fastest solution I can image. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547159#comment-15547159 ] Klaus Ma commented on MESOS-6247: - Yes, hierarchical role is the target solution for this case; do you have target date for that feature? If not, set weight to framework is the fastest solution I can image. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[
https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547134#comment-15547134
]
Benjamin Mahler commented on MESOS-6247:
Setting weight for a framework means that the FrameworkID acts a sub-role. This
would be a hacky implementation of hierarchical roles:
{noformat}
^
/ \
BigData Services
^
/ \
FrameworkId1 FrameworkId2
{noformat}
This approach only supports 2 levels of roles and the leaf nodes are not roles,
they are framework ids (which act as a special kind of role). We should support
hierarchical roles directly, supporting more than just 2 levels and having
roles as the only allocation entity:
{noformat}
^
/ \
BigData Services
^
/ \
Spark YARN
{noformat}
> Enable Framework to set weight
> --
>
> Key: MESOS-6247
> URL: https://issues.apache.org/jira/browse/MESOS-6247
> Project: Mesos
> Issue Type: Bug
> Components: allocation
> Environment: all
>Reporter: Klaus Ma
>Priority: Critical
>
> We'd like to enable framework's weight when it register. So the framework can
> share resources based on weight within the same role.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4893) Allow setting permissions and access control on persistent volumes
[ https://issues.apache.org/jira/browse/MESOS-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547132#comment-15547132 ] Greg Mann commented on MESOS-4893: -- SGTM! > Allow setting permissions and access control on persistent volumes > -- > > Key: MESOS-4893 > URL: https://issues.apache.org/jira/browse/MESOS-4893 > Project: Mesos > Issue Type: Improvement > Components: general >Reporter: Anindya Sinha >Assignee: Anindya Sinha > Labels: external-volumes, persistent-volumes > > Currently, persistent volumes are exclusive, i.e. that if a persistent volume > is used by one task or executor, it cannot be concurrently used by other task > or executor. > With the introduction of shared volumes, persistent volumes can be used > simultaneously by multiple tasks or executors. As a result, we need to > introduce setting up of ownership of persistent volumes at creation of > volumes which the tasks need to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4893) Allow setting permissions and access control on persistent volumes
[ https://issues.apache.org/jira/browse/MESOS-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547117#comment-15547117 ] Anindya Sinha commented on MESOS-4893: -- Since https://reviews.apache.org/r/46228 has some dependency (linux.cpp and posix.cpp to be specific) on https://reviews.apache.org/r/45963/ (Yan Xu is the shepherd for this also), I was hoping to have that merged first and then update this. > Allow setting permissions and access control on persistent volumes > -- > > Key: MESOS-4893 > URL: https://issues.apache.org/jira/browse/MESOS-4893 > Project: Mesos > Issue Type: Improvement > Components: general >Reporter: Anindya Sinha >Assignee: Anindya Sinha > Labels: external-volumes, persistent-volumes > > Currently, persistent volumes are exclusive, i.e. that if a persistent volume > is used by one task or executor, it cannot be concurrently used by other task > or executor. > With the introduction of shared volumes, persistent volumes can be used > simultaneously by multiple tasks or executors. As a result, we need to > introduce setting up of ownership of persistent volumes at creation of > volumes which the tasks need to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547097#comment-15547097 ] Klaus Ma commented on MESOS-6247: - Wow, here's the case I'd like to resolve: 1. There're two types of workload in the cluster: long running service & BigData workload 2. The resources (HW) for BigData workload is better 3. The BigData workload includes MapReduce (YARN) & Spark Current design is to run YARN & Spark in the same role and reserved resources for them; so I'd like to set weight for the framework. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5966) Add libprocess HTTP tests with SSL support
[ https://issues.apache.org/jira/browse/MESOS-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-5966: - Sprint: Mesosphere Sprint 40, Mesosphere Sprint 41, Mesosphere Sprint 42, Mesosphere Sprint 44 (was: Mesosphere Sprint 40, Mesosphere Sprint 41, Mesosphere Sprint 42) > Add libprocess HTTP tests with SSL support > -- > > Key: MESOS-5966 > URL: https://issues.apache.org/jira/browse/MESOS-5966 > Project: Mesos > Issue Type: Task >Reporter: Greg Mann >Assignee: Greg Mann > Labels: mesosphere > > Libprocess contains SSL unit tests which test our SSL support using simple > sockets. We should add tests which also make use of libprocess's various HTTP > classes and helpers in a variety of SSL configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3753) Test the HTTP Scheduler library with SSL enabled
[ https://issues.apache.org/jira/browse/MESOS-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-3753: - Sprint: Mesosphere Sprint 39, Mesosphere Sprint 40, Mesosphere Sprint 41, Mesosphere Sprint 42, Mesosphere Sprint 44 (was: Mesosphere Sprint 39, Mesosphere Sprint 40, Mesosphere Sprint 41, Mesosphere Sprint 42) > Test the HTTP Scheduler library with SSL enabled > > > Key: MESOS-3753 > URL: https://issues.apache.org/jira/browse/MESOS-3753 > Project: Mesos > Issue Type: Story > Components: framework, HTTP API, test >Reporter: Joseph Wu >Assignee: Greg Mann > Labels: mesosphere, security > > Currently, the HTTP Scheduler library does not support SSL-enabled Mesos. > (You can manually test this by spinning up an SSL-enabled master and attempt > to run the event-call framework example against it.) > We need to add tests that check the HTTP Scheduler library against > SSL-enabled Mesos: > * with downgrade support, > * with required framework/client-side certifications, > * with/without verification of certificates (master-side), > * with/without verification of certificates (framework-side), > * with a custom certificate authority (CA) > These options should be controlled by the same environment variables found on > the [SSL user doc|http://mesos.apache.org/documentation/latest/ssl/]. > Note: This issue will be broken down into smaller sub-issues as bugs/problems > are discovered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4893) Allow setting permissions and access control on persistent volumes
[ https://issues.apache.org/jira/browse/MESOS-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546897#comment-15546897 ] Greg Mann commented on MESOS-4893: -- Hi [~anindya.sinha], I was looking over old reviews and made a pass over your code changes in https://reviews.apache.org/r/46228/ This will require a bit of rebasing, and some re-examination in light of the pod features that have been added recently. Do you think you'll be picking this work back up? I had talked to [~xujyan] previously about reviewing these, so I'm happy to help! > Allow setting permissions and access control on persistent volumes > -- > > Key: MESOS-4893 > URL: https://issues.apache.org/jira/browse/MESOS-4893 > Project: Mesos > Issue Type: Improvement > Components: general >Reporter: Anindya Sinha >Assignee: Anindya Sinha > Labels: external-volumes, persistent-volumes > > Currently, persistent volumes are exclusive, i.e. that if a persistent volume > is used by one task or executor, it cannot be concurrently used by other task > or executor. > With the introduction of shared volumes, persistent volumes can be used > simultaneously by multiple tasks or executors. As a result, we need to > introduce setting up of ownership of persistent volumes at creation of > volumes which the tasks need to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6313) In Mesos Console, "Completed Tasks" in a tab next to "Active Tasks"
Roman Leventov created MESOS-6313: - Summary: In Mesos Console, "Completed Tasks" in a tab next to "Active Tasks" Key: MESOS-6313 URL: https://issues.apache.org/jira/browse/MESOS-6313 Project: Mesos Issue Type: Improvement Components: webui Reporter: Roman Leventov It will ease navigation between them (clicks in close areas of screen, rather than scroll) and will make "Completed Tasks" even *visible* when the list of active tasks is very long. This is important for those who are not familiar with mesos UI and expect everthing to be acessible though menus/tabs on the top of the screen, not though scrolling. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546670#comment-15546670 ] Benjamin Mahler commented on MESOS-6247: Currently the framework ID acts as a sub-role of the role of the scheduler (with no ability to set quota or weight for this sub-role). I think this was a mistake and we should have only done allocation to roles. You could imagine a model where the allocator only knows about roles, not frameworks. In this model if multiple frameworks share a role, they should co-operate within that role. So I'd say we probably want to avoid doing more sophisticated allocation between frameworks in a role. > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6312) Add requirement in upgrade.md and getting-started.md for agent '--runtime_dir' in when running as non-root
Kevin Klues created MESOS-6312:
--
Summary: Add requirement in upgrade.md and getting-started.md for
agent '--runtime_dir' in when running as non-root
Key: MESOS-6312
URL: https://issues.apache.org/jira/browse/MESOS-6312
Project: Mesos
Issue Type: Task
Reporter: Kevin Klues
Priority: Blocker
Fix For: 1.1.0
We recently introduced a new agent flag for {{--runtime_dir}}. Unlike the
{{--work_dir}}, this directory is designed to hold the state of a running agent
between subsequent agent-restarts (but not across host reboots).
By default, this flag is set to {{/var/run/mesos}} since this is a {{tempfs}}
on linux that gets automatically cleaned up on reboot. However, on most systems
{{/var/run/mesos}} is only writable by root, causing problems when launching an
agent as non-root and not pointing {{--runtime_dir}} to a different location.
We need to call this out in the upgrade.md and getting-started.md docs so that
people know they may need to set this going forward.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6177) Return unregistered agents recovered from registrar in `GetAgents` and/or `/state.json`
[ https://issues.apache.org/jira/browse/MESOS-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-6177: -- Target Version/s: 1.1.0 Fix Version/s: (was: 1.1.0) > Return unregistered agents recovered from registrar in `GetAgents` and/or > `/state.json` > --- > > Key: MESOS-6177 > URL: https://issues.apache.org/jira/browse/MESOS-6177 > Project: Mesos > Issue Type: Improvement > Components: HTTP API >Reporter: Zhitao Li >Assignee: Zhitao Li > > Use case: > This can be used for any software which talks to Mesos master to better > understand state of an unregistered agent after a master failover. > If this information is available, the use case in MESOS-6174 can be handled > with a simpler decision of whether the corresponding agent is removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-6102) Add event for agent added in master operator API
[ https://issues.apache.org/jira/browse/MESOS-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhitao Li updated MESOS-6102: - Fix Version/s: 1.1.0 > Add event for agent added in master operator API > > > Key: MESOS-6102 > URL: https://issues.apache.org/jira/browse/MESOS-6102 > Project: Mesos > Issue Type: Task >Reporter: Zhitao Li >Assignee: Zhitao Li > Fix For: 1.1.0 > > > Similar to MESOS-6101, we should support sending some event when `AgentInfo` > is added to an AgentID so subscriber can get this information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5936) Operator SUBSCRIBE api should provdide more task metadata than just state changes
[ https://issues.apache.org/jira/browse/MESOS-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhitao Li updated MESOS-5936: - Shepherd: Anand Mazumdar > Operator SUBSCRIBE api should provdide more task metadata than just state > changes > - > > Key: MESOS-5936 > URL: https://issues.apache.org/jira/browse/MESOS-5936 > Project: Mesos > Issue Type: Improvement > Components: HTTP API, json api >Affects Versions: 1.0.0 >Reporter: Steven Schlansker >Assignee: Zhitao Li > Labels: mesosphere > Fix For: 1.1.0 > > > I am starting to use the new Operator event subscription API to get notified > of task changes. The initial TASK_STAGING event has a good amount of > information, but unfortunately future events like TASK_RUNNING include almost > no metadata. This means that any task information that cannot be determined > until the task launches (in my particular case, the IP address assigned by > the Docker containerizer) is not available through the event stream. > Here is a gist of a single task that was launched, comparing the information > in 'state.json' vs the subscribed events: > https://gist.github.com/stevenschlansker/c1d32aa9ce37a73f9c4d64347397d3b8 > Note particularly how the IP address never shows in the event stream. > Task updates should expose the task information that changed. If this is too > onerous, maybe the subscription call can take optional sets of data to > include, with the first one being additional task metadata. > A possible workaround is to use the task events to fetch 'state.json' > separately, but this is inherently racy and totally undermines the utility of > the event stream api. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-6177) Return unregistered agents recovered from registrar in `GetAgents` and/or `/state.json`
[ https://issues.apache.org/jira/browse/MESOS-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhitao Li updated MESOS-6177: - Shepherd: Anand Mazumdar (was: Vinod Kone) Fix Version/s: 1.1.0 > Return unregistered agents recovered from registrar in `GetAgents` and/or > `/state.json` > --- > > Key: MESOS-6177 > URL: https://issues.apache.org/jira/browse/MESOS-6177 > Project: Mesos > Issue Type: Improvement > Components: HTTP API >Reporter: Zhitao Li >Assignee: Zhitao Li > Fix For: 1.1.0 > > > Use case: > This can be used for any software which talks to Mesos master to better > understand state of an unregistered agent after a master failover. > If this information is available, the use case in MESOS-6174 can be handled > with a simpler decision of whether the corresponding agent is removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6311) Consider supporting implicit reconciliation per agent
[ https://issues.apache.org/jira/browse/MESOS-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546209#comment-15546209 ] Joris Van Remoortere commented on MESOS-6311: - cc [~anandmazumdar] [~neilconway] [~vinodkone] > Consider supporting implicit reconciliation per agent > - > > Key: MESOS-6311 > URL: https://issues.apache.org/jira/browse/MESOS-6311 > Project: Mesos > Issue Type: Improvement > Components: master >Reporter: Joris Van Remoortere > > Currently mesos only supports: > - total implicit reconciliation > - explicit reconciliation per task > Since agent can slowly rejoin the master after a master failover, it is hard > to have a low time bound on implicit reconciliation for tasks. > Performing the current implicit reconciliation is expensive on big clusters > so it should not be done every N seconds. > If we could perform implicit reconciliation for a particular agent, then it > would be cheap enough to after we notice that particular agent rejoining the > cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6311) Consider supporting implicit reconciliation per agent
Joris Van Remoortere created MESOS-6311: --- Summary: Consider supporting implicit reconciliation per agent Key: MESOS-6311 URL: https://issues.apache.org/jira/browse/MESOS-6311 Project: Mesos Issue Type: Improvement Components: master Reporter: Joris Van Remoortere Currently mesos only supports: - total implicit reconciliation - explicit reconciliation per task Since agent can slowly rejoin the master after a master failover, it is hard to have a low time bound on implicit reconciliation for tasks. Performing the current implicit reconciliation is expensive on big clusters so it should not be done every N seconds. If we could perform implicit reconciliation for a particular agent, then it would be cheap enough to after we notice that particular agent rejoining the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-6283) Fix the Web UI allowing access to the task sandbox for nested containers.
[ https://issues.apache.org/jira/browse/MESOS-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15540415#comment-15540415 ] haosdent edited comment on MESOS-6283 at 10/4/16 6:02 PM: -- | Filled missing executor info in tasks when `LAUNCH_GROUP`. | https://reviews.apache.org/r/52470/ | | Exposed the executor's type in the endpoints. | https://reviews.apache.org/r/52520/ | | Fixed the wrong sandbox directory of the tasks in Web UI. | https://reviews.apache.org/r/52471 | was (Author: haosd...@gmail.com): | Filled missing executor info in tasks when `LAUNCH_GROUP`. | https://reviews.apache.org/r/52470/ | | Fixed the wrong sandbox directory of the tasks in Web UI. | https://reviews.apache.org/r/52471 | > Fix the Web UI allowing access to the task sandbox for nested containers. > - > > Key: MESOS-6283 > URL: https://issues.apache.org/jira/browse/MESOS-6283 > Project: Mesos > Issue Type: Bug > Components: webui >Reporter: Anand Mazumdar >Assignee: haosdent > Labels: mesosphere > Fix For: 1.1.0 > > Attachments: sandbox.gif > > > Currently, the sandbox button for a child task is broken on the WebUI. It > does nothing and dies with an error that the executor for this task cannot be > found. We need to fix the WebUI to follow the symlink "tasks/taskId" and > display the task sandbox to the users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-6304) Add authentication support to the default executor
[ https://issues.apache.org/jira/browse/MESOS-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-6304: - Summary: Add authentication support to the default executor (was: Add authentication support for the default executor) > Add authentication support to the default executor > -- > > Key: MESOS-6304 > URL: https://issues.apache.org/jira/browse/MESOS-6304 > Project: Mesos > Issue Type: Improvement >Reporter: Galen Pewtherer >Assignee: Artem Harutyunyan > > Right now the default executor (used to launch task groups) does not > authenticate with either the executor API (/v1/executor) or the agent API > (v1). Ofcourse, the driver based executor doesn't authenticate either. > It would be great to come up with a solution that works for both the built-in > executors and custom executors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6100) Make fails compiling 1.0.1
[
https://issues.apache.org/jira/browse/MESOS-6100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546107#comment-15546107
]
Kevin Klues commented on MESOS-6100:
This was fixed by: https://reviews.apache.org/r/52113
[~neilc] Can you add a backlink to this bug in the review.
{noformat}
commit 84a3adb92e53a5683584df84971272d840cbe96b
Author: Neil Conway
Date: Wed Sep 21 10:23:47 2016 -0700
Fixed an uninitialized variable warning.
Observed with clang-tidy.
Review: https://reviews.apache.org/r/52113/
{noformat}
> Make fails compiling 1.0.1
> ---
>
> Key: MESOS-6100
> URL: https://issues.apache.org/jira/browse/MESOS-6100
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: Alpine Linux (Edge)
> GCC 6.1.1
>Reporter: Gennady Feldman
>Assignee: Kevin Klues
>
> linux/fs.cpp: In static member function 'static
> Try
> mesos::internal::fs::MountInfoTable::read(const Option&, bool)':
> linux/fs.cpp:152:27: error: 'rootParentId' may be used uninitialized in this
> function [-Werror=maybe-uninitialized]
> sortFrom(rootParentId);
>^
> cc1plus: all warnings being treated as errors
> P.S. This is something new since I am able to compile 1.0.0 just fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Klues updated MESOS-6310:
---
Shepherd: Jie Yu (was: Kevin Klues)
Target Version/s: 1.1.0
Fix Version/s: 1.1.0
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Assignee: Kevin Klues
>Priority: Minor
> Fix For: 1.1.0
>
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Klues reassigned MESOS-6310:
--
Assignee: Kevin Klues
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Assignee: Kevin Klues
>Priority: Minor
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545829#comment-15545829
]
Kevin Klues commented on MESOS-6310:
Since it is not POSIX, it seems reasonable to add something like:
{noformat}
#ifndef W_EXITCODE
#define W_EXITCODE ...
#endif
{noformat}
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Priority: Minor
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6207) Python bindings fail to build with custom SVN installation path
[
https://issues.apache.org/jira/browse/MESOS-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545787#comment-15545787
]
Ilya Pronin commented on MESOS-6207:
Thanks!
> Python bindings fail to build with custom SVN installation path
> ---
>
> Key: MESOS-6207
> URL: https://issues.apache.org/jira/browse/MESOS-6207
> Project: Mesos
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.1
>Reporter: Ilya Pronin
>Assignee: Ilya Pronin
>Priority: Trivial
>
> In {{src/Makefile.am}} {{PYTHON_LDFLAGS}} variable is used while building
> Python bindings. This variable picks {{LDFLAGS}} during configuration phase
> before we check for custom SVN installation path and misses
> {{-L$\{with_svn\}/lib}} flag. That causes a link error on systems with
> uncommon SVN installation path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545775#comment-15545775
]
haosdent commented on MESOS-6310:
-
{{sys/wait.h}} should have definition for this
https://android.googlesource.com/platform/bionic/+/d49850d/libc/include/sys/wait.h#48
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Priority: Minor
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3157) only perform batch resource allocations
[
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423829#comment-15423829
]
Jacob Janco edited comment on MESOS-3157 at 10/4/16 3:14 PM:
-
New reviews:
changes to allocator: https://reviews.apache.org/r/51027/
fixes to tests: https://reviews.apache.org/r/51028/
was (Author: jjanco):
New reviews:
changes to allocator: https://reviews.apache.org/r/51027/
benchmark: https://reviews.apache.org/r/49617/
fixes to tests: https://reviews.apache.org/r/51028/
> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
> Issue Type: Bug
> Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live
> frameworks that often revive offers. Running the allocator takes a long time
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the
> allocator process to get very long, and the allocator effectively becomes
> unresponsive (eg. a revive offers message takes too long to come to the head
> of the queue).
> We have been running a patch to remove all the event-triggered allocations
> and only allocate from the batch task
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves
> responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (MESOS-3157) only perform batch resource allocations
[
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacob Janco updated MESOS-3157:
---
Comment: was deleted
(was: Some interesting output from the benchmark listed in the reviews:
Sample output without 51027:
[ RUN ]
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 57251us
Added 1 agents in 3.2134535333mins
allocator settled after 1.6123603833mins
[ OK ]
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(290578 ms)
Sample output with 51027:
[ RUN ]
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 39817us
Added 1 agents in 3.2286054167mins
allocator settled after 25.525654secs
[ OK ]
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
(220137 ms)
Any input on the benchmark would be greatly appreciated as well. Thanks!)
> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
> Issue Type: Bug
> Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live
> frameworks that often revive offers. Running the allocator takes a long time
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the
> allocator process to get very long, and the allocator effectively becomes
> unresponsive (eg. a revive offers message takes too long to come to the head
> of the queue).
> We have been running a patch to remove all the event-triggered allocations
> and only allocate from the batch task
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves
> responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545503#comment-15545503
]
ASF GitHub Bot commented on MESOS-6310:
---
Github user h0tbird commented on the pull request:
https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20#commitcomment-19287343
In src/slave/containerizer/mesos/containerizer.cpp:
In src/slave/containerizer/mesos/containerizer.cpp on line 1945:
Done: [MESOS-6310](https://issues.apache.org/jira/browse/MESOS-6310)
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Priority: Minor
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6310) Remove or define non-POSIX function
[
https://issues.apache.org/jira/browse/MESOS-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marc Villacorta updated MESOS-6310:
---
Summary: Remove or define non-POSIX function (was: Remove or define
non-posix function)
> Remove or define non-POSIX function
> ---
>
> Key: MESOS-6310
> URL: https://issues.apache.org/jira/browse/MESOS-6310
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Affects Versions: 1.0.2
>Reporter: Marc Villacorta
>Priority: Minor
>
> I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
> But this [commit|
> https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
> introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and
> seems to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-6310) Remove or define non-posix function
Marc Villacorta created MESOS-6310:
--
Summary: Remove or define non-posix function
Key: MESOS-6310
URL: https://issues.apache.org/jira/browse/MESOS-6310
Project: Mesos
Issue Type: Improvement
Components: containerization
Affects Versions: 1.0.2
Reporter: Marc Villacorta
Priority: Minor
I was trying to compile Mesos using _musl_ inside Alpine Linux 3.4.
But this [commit|
https://github.com/apache/mesos/commit/498d14e934233e4501597b43da3924bfe8b2de20]
introduced the {{W_EXITCODE()}} macro which is not defined in _musl_ and seems
to be non-POSIX.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-6309) Mesos-specific targets appear in libprocess' cmake config.
Alexander Rukletsov created MESOS-6309:
--
Summary: Mesos-specific targets appear in libprocess' cmake config.
Key: MESOS-6309
URL: https://issues.apache.org/jira/browse/MESOS-6309
Project: Mesos
Issue Type: Improvement
Components: build
Affects Versions: 1.0.0
Reporter: Alexander Rukletsov
File
https://github.com/apache/mesos/blob/caaa98bd6e0a3a80f10a94e320d4581883f8453c/3rdparty/libprocess/cmake/Process3rdpartyConfigure.cmake
defines some Mesos-related targets, e.g. {{MESOS_FETCHER}} that do not belong
in libprocess.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6216) LibeventSSLSocketImpl::create is not safe to call concurrently with os::getenv
[
https://issues.apache.org/jira/browse/MESOS-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Bannier updated MESOS-6216:
Target Version/s: 1.1.0, 1.0.2
> LibeventSSLSocketImpl::create is not safe to call concurrently with os::getenv
> --
>
> Key: MESOS-6216
> URL: https://issues.apache.org/jira/browse/MESOS-6216
> Project: Mesos
> Issue Type: Bug
> Components: security
>Reporter: Benjamin Bannier
>Assignee: Benjamin Bannier
> Labels: mesosphere
> Attachments: build.log
>
>
> {{LibeventSSLSocketImpl::create}} is called whenever a potentially
> ssl-enabled socket is created. It in turn calls {{openssl::initialize}} which
> calls a function {{reinitialize}} using {{os::setenv}}. Here {{os::setenv}}
> is used to set up SSL-related libprocess environment variables
> {{LIBPROCESS_SSL_*}}.
> Since {{os::setenv}} is not thread-safe just like the {{::setenv}} it wraps,
> any calling of functions like {{os::getenv}} (or via {{os::environment}})
> concurrently with the first invocation of {{LibeventSSLSocketImpl::create}}
> performs unsynchronized r/w access to the same data structure in the runtime.
> We usually perform most setup of the environment before we start the
> libprocess runtime with {{process::initialize}} from a {{main}} function, see
> e.g., {{src/slave/main.cpp}} or {{src/master/main.cpp}} and others. It
> appears that we should move the setup of libprocess' SSL environment
> variables to a similar spot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6247) Enable Framework to set weight
[ https://issues.apache.org/jira/browse/MESOS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544801#comment-15544801 ] Klaus Ma commented on MESOS-6247: - [~bmahler], any suggestion? > Enable Framework to set weight > -- > > Key: MESOS-6247 > URL: https://issues.apache.org/jira/browse/MESOS-6247 > Project: Mesos > Issue Type: Bug > Components: allocation > Environment: all >Reporter: Klaus Ma >Priority: Critical > > We'd like to enable framework's weight when it register. So the framework can > share resources based on weight within the same role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4775) Add allocator metric for number of active inverse offer filters
[ https://issues.apache.org/jira/browse/MESOS-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier updated MESOS-4775: Assignee: (was: Benjamin Bannier) > Add allocator metric for number of active inverse offer filters > > > Key: MESOS-4775 > URL: https://issues.apache.org/jira/browse/MESOS-4775 > Project: Mesos > Issue Type: Improvement > Components: allocation >Reporter: Benjamin Bannier > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
