[jira] [Comment Edited] (MESOS-6845) All /master endpoint documentation lack request details
[ https://issues.apache.org/jira/browse/MESOS-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795623#comment-15795623 ] Joerg Schad edited comment on MESOS-6845 at 1/3/17 5:53 PM: Agree. Just in the above case, the operator v1 API does not support `teardown` (added https://issues.apache.org/jira/browse/MESOS-6846 to track that). was (Author: js84): Agree. Just in the above case, the operator v1 API does not support `teardown`. > All /master endpoint documentation lack request details > --- > > Key: MESOS-6845 > URL: https://issues.apache.org/jira/browse/MESOS-6845 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Tobias Mueller > > When I was trying to use the /master/teardown endpoint > (http://mesos.apache.org/documentation/latest/endpoints/master/teardown/), I > was unsuccessful in tearing down my specific framework, because the docs say > nothing about request details, such as > * HTTP method > * Request body contents > This makes the usage rather hard, if not impossible. > I'd suggest to add call examples like for the Scheduler HTTP API at > http://mesos.apache.org/documentation/latest/scheduler-http-api/ > E.g. > TEARDOWN Request (JSON): > POST /master/teardown HTTP/1.1 > Host: masterhost:5050 > Content-Type: application/json > frameworkId=12220-3440-12532-2345 > TEARDOWN Response: > HTTP/1.1 200 Ok -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6846) Support `teardown` in the v1 operator API.
Joerg Schad created MESOS-6846: -- Summary: Support `teardown` in the v1 operator API. Key: MESOS-6846 URL: https://issues.apache.org/jira/browse/MESOS-6846 Project: Mesos Issue Type: Improvement Components: HTTP API Reporter: Joerg Schad Currently, the v1 operator API does not support teardown of frameworks. The semantics should be similar to the old HTTP endpoint: http://mesos.apache.org/documentation/latest/endpoints/master/teardown/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6845) All /master endpoint documentation lack request details
[ https://issues.apache.org/jira/browse/MESOS-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795623#comment-15795623 ] Joerg Schad commented on MESOS-6845: Agree. Just in the above case, the operator v1 API does not support `teardown`. > All /master endpoint documentation lack request details > --- > > Key: MESOS-6845 > URL: https://issues.apache.org/jira/browse/MESOS-6845 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Tobias Mueller > > When I was trying to use the /master/teardown endpoint > (http://mesos.apache.org/documentation/latest/endpoints/master/teardown/), I > was unsuccessful in tearing down my specific framework, because the docs say > nothing about request details, such as > * HTTP method > * Request body contents > This makes the usage rather hard, if not impossible. > I'd suggest to add call examples like for the Scheduler HTTP API at > http://mesos.apache.org/documentation/latest/scheduler-http-api/ > E.g. > TEARDOWN Request (JSON): > POST /master/teardown HTTP/1.1 > Host: masterhost:5050 > Content-Type: application/json > frameworkId=12220-3440-12532-2345 > TEARDOWN Response: > HTTP/1.1 200 Ok -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513045#comment-15513045 ] Joerg Schad commented on MESOS-5070: The discussion seems to divert from the original purpose of this ticket. I would close this ticket for now. Feel free to reopen if you disagree... > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510435#comment-15510435 ] Joerg Schad edited comment on MESOS-5070 at 9/21/16 4:51 PM: - The current state is that they only work on linux (windows also ignores the explicit options right now and as you pointed out does not care about parentHooks). If we are are talking about adding windows support in the future childHooks have one advantage over parentHooks in my opinion: While parentHooks can be arbitrary functions, childHooks are constrained (via the factory methods) to a small set of predefined hooks to which we could add some form of id and enable windows to implement its own version of it. Maybe [~jieyu] can add more details here. was (Author: js84): The current state is that they only work on linux (windows also ignores the explicit options right now and as you pointed out does not care about parentHooks). If we are are talking about adding windows support in the future childHooks have one advantage over parentHooks in my opinion: While parentHooks can be arbitrary functions, childHooks are constrained (via the factory methods) to a small set of predefined hooks -to which we could add some form of id and enable windows to implement its own version of it-. Maybe [~jieyu] can add more details here. > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510435#comment-15510435 ] Joerg Schad commented on MESOS-5070: The current state is that they only work on linux (windows also ignores the explicit options right now and as you pointed out does not care about parentHooks). If we are are talking about adding windows support in the future childHooks have one advantage over parentHooks in my opinion: While parentHooks can be arbitrary functions, childHooks are constrained (via the factory methods) to a small set of predefined hooks -to which we could add some form of id and enable windows to implement its own version of it-. Maybe [~jieyu] can add more details here. > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-6153) Resource leak in slave.cpp.
[
https://issues.apache.org/jira/browse/MESOS-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-6153:
---
Summary: Resource leak in slave.cpp. (was: Resource leak in slave.coo.)
> Resource leak in slave.cpp.
> ---
>
> Key: MESOS-6153
> URL: https://issues.apache.org/jira/browse/MESOS-6153
> Project: Mesos
> Issue Type: Bug
>Reporter: Joerg Schad
>Assignee: Benjamin Bannier
> Labels: coverity
>
> Coverity detected the following resource leak:
> {code}
> 1. Condition this->queuedTasks.contains(taskId), taking true branch.
> 6547 if (queuedTasks.contains(taskId)) {
> 2. Condition terminal, taking true branch.
> 6548if (terminal) {
> 3. alloc_fn: Storage is returned from allocation function operator new.
> [Note: The source code implementation of the function has been overridden by
> a builtin model.]
> 4. var_assign: Assigning: task = storage returned from new
> mesos::Task(mesos::internal::protobuf::createTask(this->queuedTasks.at(taskId),
> mesos::TaskState const(status->state()), this->frameworkId)).
> 6549 Task* task = new Task(protobuf::createTask(
> 6550 queuedTasks.at(taskId),
> 6551 status.state(),
> 6552 frameworkId));
> 6553
> 6554 queuedTasks.erase(taskId);
> 6555
> 6556 // This might be a queued task belonging to a task group.
> 6557 // If so, we need to update the other tasks belonging to this task
> group.
> 6558 Option taskGroup = getQueuedTaskGroup(taskId);
> 6559
> 5. Condition taskGroup.isSome(), taking true branch.
> 6560 if (taskGroup.isSome()) {
> 6. No elements left in taskGroup->tasks(), leaving loop.
> 6561foreach (const TaskInfo& task_, taskGroup->tasks()) {
> 6562 Task* task = new Task(
> 6563 protobuf::createTask(task_, status.state(), frameworkId));
> 6564
> 6565 tasks.push_back(task);
> 6566}
> 7. Falling through to end of if statement.
> 6567 } else {
> 6568tasks.push_back(task);
> 6569 }
>
> CID 1372871 (#1 of 1): Resource leak (RESOURCE_LEAK)
> 8. leaked_storage: Variable task going out of scope leaks the storage it
> points to.
> 6570} else {
> 6571 return Error("Cannot send non-terminal update for queued task");
> {code}
> https://scan5.coverity.com/reports.htm#v39597/p10429/fileInstanceId=98881751=28450463
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-6153) Resource leak in slave.coo.
Joerg Schad created MESOS-6153:
--
Summary: Resource leak in slave.coo.
Key: MESOS-6153
URL: https://issues.apache.org/jira/browse/MESOS-6153
Project: Mesos
Issue Type: Bug
Reporter: Joerg Schad
Assignee: Benjamin Bannier
Coverity detected the following resource leak:
{code}
1. Condition this->queuedTasks.contains(taskId), taking true branch.
6547 if (queuedTasks.contains(taskId)) {
2. Condition terminal, taking true branch.
6548if (terminal) {
3. alloc_fn: Storage is returned from allocation function operator new.
[Note: The source code implementation of the function has been overridden by a
builtin model.]
4. var_assign: Assigning: task = storage returned from new
mesos::Task(mesos::internal::protobuf::createTask(this->queuedTasks.at(taskId),
mesos::TaskState const(status->state()), this->frameworkId)).
6549 Task* task = new Task(protobuf::createTask(
6550 queuedTasks.at(taskId),
6551 status.state(),
6552 frameworkId));
6553
6554 queuedTasks.erase(taskId);
6555
6556 // This might be a queued task belonging to a task group.
6557 // If so, we need to update the other tasks belonging to this task
group.
6558 Option taskGroup = getQueuedTaskGroup(taskId);
6559
5. Condition taskGroup.isSome(), taking true branch.
6560 if (taskGroup.isSome()) {
6. No elements left in taskGroup->tasks(), leaving loop.
6561foreach (const TaskInfo& task_, taskGroup->tasks()) {
6562 Task* task = new Task(
6563 protobuf::createTask(task_, status.state(), frameworkId));
6564
6565 tasks.push_back(task);
6566}
7. Falling through to end of if statement.
6567 } else {
6568tasks.push_back(task);
6569 }
CID 1372871 (#1 of 1): Resource leak (RESOURCE_LEAK)
8. leaked_storage: Variable task going out of scope leaks the storage it points
to.
6570} else {
6571 return Error("Cannot send non-terminal update for queued task");
{code}
https://scan5.coverity.com/reports.htm#v39597/p10429/fileInstanceId=98881751=28450463
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-6152) Resource leak in libevent_ssl_socket.cpp.
Joerg Schad created MESOS-6152:
--
Summary: Resource leak in libevent_ssl_socket.cpp.
Key: MESOS-6152
URL: https://issues.apache.org/jira/browse/MESOS-6152
Project: Mesos
Issue Type: Bug
Reporter: Joerg Schad
Assignee: Benjamin Bannier
Coverity detected the following resource leak.
IMO {code} if (fd == -1) {code} should be {code} if (owned_fd == -1) {code}.
{code}
// Duplicate the file descriptor because Libevent will take ownership
754 // and control the lifecycle separately.
755 //
756 // TODO(josephw): We can avoid duplicating the file descriptor in
757 // future versions of Libevent. In Libevent versions 2.1.2 and later,
758 // we may use `evbuffer_file_segment_new` and `evbuffer_add_file_segment`
759 // instead of `evbuffer_add_file`.
3. open_fn: Returning handle opened by dup.
4. var_assign: Assigning: owned_fd = handle returned from dup(fd).
760 int owned_fd = dup(fd);
CID 1372873: Argument cannot be negative (REVERSE_NEGATIVE) [select
issue]
5. Condition fd == -1, taking true branch.
761 if (fd == -1) {
CID 1372872 (#1 of 1): Resource leak (RESOURCE_LEAK)
6. leaked_handle: Handle variable owned_fd going out of scope leaks the handle.
762return Failure(ErrnoError("Failed to duplicate file descriptor"));
763 }
{code}
https://scan5.coverity.com/reports.htm#v39597/p10429/fileInstanceId=98881747=28450468
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[
https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481886#comment-15481886
]
Joerg Schad edited comment on MESOS-5070 at 9/11/16 5:37 PM:
-
1.
Fair, do you feel it needs to be part of this reviews?
2.
Watchdog is a different concept than a ChildHook:
A ChildHook is a function which is executed before the actual function, where
the watchdog process is running during the entire lifetime of the process.
{code}
// Creates a seperate watchdog process to monitor the child process and
// kill it in case the parent process dies.
{code}
I would be happy to rename it 'Supervisor', but in a different review :-).
was (Author: js84):
1.
Where do you want to rename Hook -> ParentHook? If you refer to
Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no
reason to rename it.
2.
Watchdog is a different concept than a ChildHook:
A ChildHook is a function which is executed before the actual function, where
the watchdog process is running during the entire lifetime of the process.
{code}
// Creates a seperate watchdog process to monitor the child process and
// kill it in case the parent process dies.
{code}
I would be happy to rename it 'Supervisor', but in a different review :-).
> Introduce more flexible subprocess interface for child options.
> ---
>
> Key: MESOS-5070
> URL: https://issues.apache.org/jira/browse/MESOS-5070
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
> Labels: tech-debt
>
> We introduced a number of parameters to the subprocess interface with
> MESOS-5049.
> Adding all options explicitly to the subprocess interface makes it
> inflexible.
> We should investigate a flexible options, which still prevents arbitrary code
> to be executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5070: --- Comment: was deleted (was: 1. Where do you want to rename Hook -> ParentHook? If you refer to Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no reason to rename it. 2. Watchdog is a different concept than a ChildHook: A ChildHook is a function which is executed before the actual function, where the watchdog process is running during the entire lifetime of the process. [code] // Creates a seperate watchdog process to monitor the child process and // kill it in case the parent process dies. [code] I would be happy to rename it 'Supervisor', but in a different review :-).) > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[
https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481886#comment-15481886
]
Joerg Schad edited comment on MESOS-5070 at 9/11/16 3:06 PM:
-
1.
Where do you want to rename Hook -> ParentHook? If you refer to
Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no
reason to rename it.
2.
Watchdog is a different concept than a ChildHook:
A ChildHook is a function which is executed before the actual function, where
the watchdog process is running during the entire lifetime of the process.
{code}
// Creates a seperate watchdog process to monitor the child process and
// kill it in case the parent process dies.
{code}
I would be happy to rename it 'Supervisor', but in a different review :-).
was (Author: js84):
1.
Where do you want to rename Hook -> ParentHook? If you refer to
Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no
reason to rename it.
2.
Watchdog is a different concept than a ChildHook:
A ChildHook is a function which is executed before the actual function, where
the watchdog process is running during the entire lifetime of the process.
[code]
// Creates a seperate watchdog process to monitor the child process and
// kill it in case the parent process dies.
[code]
I would be happy to rename it 'Supervisor', but in a different review :-).
> Introduce more flexible subprocess interface for child options.
> ---
>
> Key: MESOS-5070
> URL: https://issues.apache.org/jira/browse/MESOS-5070
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
> Labels: tech-debt
>
> We introduced a number of parameters to the subprocess interface with
> MESOS-5049.
> Adding all options explicitly to the subprocess interface makes it
> inflexible.
> We should investigate a flexible options, which still prevents arbitrary code
> to be executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481886#comment-15481886 ] Joerg Schad commented on MESOS-5070: 1. Where do you want to rename Hook -> ParentHook? If you refer to Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no reason to rename it. 2. Watchdog is a different concept than a ChildHook: A ChildHook is a function which is executed before the actual function, where the watchdog process is running during the entire lifetime of the process. [code] // Creates a seperate watchdog process to monitor the child process and // kill it in case the parent process dies. [code] I would be happy to rename it 'Supervisor', but in a different review :-). > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481887#comment-15481887 ] Joerg Schad commented on MESOS-5070: 1. Where do you want to rename Hook -> ParentHook? If you refer to Subprocess::Hook this is used for both ParentHooks and ChildHooks, so I see no reason to rename it. 2. Watchdog is a different concept than a ChildHook: A ChildHook is a function which is executed before the actual function, where the watchdog process is running during the entire lifetime of the process. [code] // Creates a seperate watchdog process to monitor the child process and // kill it in case the parent process dies. [code] I would be happy to rename it 'Supervisor', but in a different review :-). > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5070) Introduce more flexible subprocess interface for child options.
[ https://issues.apache.org/jira/browse/MESOS-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433343#comment-15433343 ] Joerg Schad commented on MESOS-5070: [~jieyu] I am happy to reopen/rebase this if you agree with the overall structure > Introduce more flexible subprocess interface for child options. > --- > > Key: MESOS-5070 > URL: https://issues.apache.org/jira/browse/MESOS-5070 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: tech-debt > > We introduced a number of parameters to the subprocess interface with > MESOS-5049. > Adding all options explicitly to the subprocess interface makes it > inflexible. > We should investigate a flexible options, which still prevents arbitrary code > to be executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5984) Add coverity build to ASF CI.
[ https://issues.apache.org/jira/browse/MESOS-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5984: --- Summary: Add coverity build to ASF CI. (was: Add coverity build to ASF ci.) > Add coverity build to ASF CI. > - > > Key: MESOS-5984 > URL: https://issues.apache.org/jira/browse/MESOS-5984 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > > We should enable a daily (not more frequent, due to limits of the coverity > scan service) coverity build on the ASF infrastructure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5984) Add coverity build to ASF ci.
Joerg Schad created MESOS-5984: -- Summary: Add coverity build to ASF ci. Key: MESOS-5984 URL: https://issues.apache.org/jira/browse/MESOS-5984 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad Assignee: Joerg Schad We should enable a daily (not more frequent, due to limits of the coverity scan service) coverity build on the ASF infrastructure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5862) External links to .md files broken.
[ https://issues.apache.org/jira/browse/MESOS-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399023#comment-15399023 ] Joerg Schad commented on MESOS-5862: This is actually a regression: Consider the sandbox link here: http://mesos.apache.org/documentation/latest/docker-volume/ > External links to .md files broken. > --- > > Key: MESOS-5862 > URL: https://issues.apache.org/jira/browse/MESOS-5862 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad >Assignee: Joerg Schad > > The Rakefile for parsing seems to parse some links to external .md links > incorrectly. See for example Apcc link here : > http://mesos.apache.org/documentation/latest/container-image/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5862) External links to .md files broken.
[ https://issues.apache.org/jira/browse/MESOS-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad reassigned MESOS-5862: -- Assignee: Joerg Schad > External links to .md files broken. > --- > > Key: MESOS-5862 > URL: https://issues.apache.org/jira/browse/MESOS-5862 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad >Assignee: Joerg Schad > > The Rakefile for parsing seems to parse some links to external .md links > incorrectly. See for example Apcc link here : > http://mesos.apache.org/documentation/latest/container-image/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5862) External links to .md files broken.
Joerg Schad created MESOS-5862: -- Summary: External links to .md files broken. Key: MESOS-5862 URL: https://issues.apache.org/jira/browse/MESOS-5862 Project: Mesos Issue Type: Bug Reporter: Joerg Schad The Rakefile for parsing seems to parse some links to external .md links incorrectly. See for example Apcc link here : http://mesos.apache.org/documentation/latest/container-image/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5757: --- Sprint: Mesosphere Sprint 38 > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone >Assignee: Joerg Schad >Priority: Critical > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364169#comment-15364169 ] Joerg Schad commented on MESOS-5757: Removed CHECK from orphan_task filtering. https://reviews.apache.org/r/49697/ Renamed incorrect variable name in authorization tests. https://reviews.apache.org/r/49698/ Changed name of authorization test. https://reviews.apache.org/r/49699/ Added test to check orphaned tasks are filtered. https://reviews.apache.org/r/49700/ > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone >Assignee: Joerg Schad >Priority: Critical > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361699#comment-15361699 ] Joerg Schad edited comment on MESOS-5757 at 7/5/16 3:06 PM: Extended install() to support 7 arguments. https://reviews.apache.org/r/49606/ Added support for recovered frameworks. https://reviews.apache.org/r/49607/ Added filtering for orphaned tasks in /state endpoint. https://reviews.apache.org/r/49609/ Test https://reviews.apache.org/r/49639/ was (Author: js84): Extended install() to support 7 arguments. https://reviews.apache.org/r/49606/ Added support for recovered frameworks. https://reviews.apache.org/r/49607/ Added filtering for orphaned tasks in /state endpoint. https://reviews.apache.org/r/49609/ (test are almost ready and will follow tomorrow) > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone >Assignee: Joerg Schad > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361699#comment-15361699 ] Joerg Schad commented on MESOS-5757: Extended install() to support 7 arguments. https://reviews.apache.org/r/49606/ Added support for recovered frameworks. https://reviews.apache.org/r/49607/ Added filtering for orphaned tasks in /state endpoint. https://reviews.apache.org/r/49609/ (test are almost ready and will follow tomorrow) > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone >Assignee: Joerg Schad > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad reassigned MESOS-5757: -- Assignee: Joerg Schad > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone >Assignee: Joerg Schad > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360603#comment-15360603 ] Joerg Schad commented on MESOS-5709: Refactored /role and getRoles endpoint code. https://reviews.apache.org/r/49574/ Fixed `delete` usage in authorization_test.cpp. https://reviews.apache.org/r/49575/ > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Blocker > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5757) Authorize orphaned tasks
[ https://issues.apache.org/jira/browse/MESOS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360292#comment-15360292 ] Joerg Schad commented on MESOS-5757: The problem with orphaned tasks is that we don't necessarily access to the FrameworkInfo and hence we cannot authorize them as we would do for running/staged task (i.e., first check Task/TaskInfo and as fallback check FrameworInfo). I see three options for dealing with this problem: 1. Coarse granular authz for orphaned tasks Introduce a new acls/authz action for allowing a user to view orphaned tasks (i.e., all or none). 2. Add FrameworkInfo to Task (and then authorize orphaned Tasks the same way we would running/staged ones) The drawback here would be the increased size (and hence memory usage) for storing Tasks. FrameworkInfo can have large fields. Note that FrameworkInfo will be stored for each Task and hence potentially duplicated. 3. Add a new datastructure to the master storing a mapping of frameworkID to frameworkInfo (the frameworkInfo is part of Task and hence we can authorize orphaned Tasks the same way we would running/staged ones) This would lead to less storage overhead compared to 2, but might be a little more complex to implement as we have to fill/cleanup such datastructure. > Authorize orphaned tasks > > > Key: MESOS-5757 > URL: https://issues.apache.org/jira/browse/MESOS-5757 > Project: Mesos > Issue Type: Bug > Components: security >Affects Versions: 1.0.0 >Reporter: Vinod Kone > Labels: mesosphere, security > Fix For: 1.0.0 > > > Currently, orphaned tasks are not filtered (i.e., using authorization) when a > request is made to /state endpoint. This is inconsistent (and unexpected) > with how we filter un-orphaned tasks. > This is tricky because master and hence the authorizer do not have > FrameworkInfos for these orphaned tasks, until after the corresponding > frameworks re-register. > One option is for the agent to include FrameworkInfos of all its tasks and > executors in its re-registration message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5759) ProcessRemoteLinkTest.RemoteUseStaleLink and RemoteStaleLinkRelink are flaky
[
https://issues.apache.org/jira/browse/MESOS-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358803#comment-15358803
]
Joerg Schad commented on MESOS-5759:
I actually had all those test fail on head on OSx
[ FAILED ] 6 tests, listed below:
[ FAILED ] ProcessRemoteLinkTest.RemoteLink
[ FAILED ] ProcessRemoteLinkTest.RemoteRelink
[ FAILED ] ProcessRemoteLinkTest.RemoteLinkRelink
[ FAILED ] ProcessRemoteLinkTest.RemoteDoubleLinkRelink
[ FAILED ] ProcessRemoteLinkTest.RemoteUseStaleLink
[ FAILED ] ProcessRemoteLinkTest.RemoteStaleLinkRelink
6 FAILED TESTS
> ProcessRemoteLinkTest.RemoteUseStaleLink and RemoteStaleLinkRelink are flaky
>
>
> Key: MESOS-5759
> URL: https://issues.apache.org/jira/browse/MESOS-5759
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
>Affects Versions: 1.0.0
>Reporter: Joseph Wu
>Assignee: Joseph Wu
> Labels: libprocess, mesosphere
>
> {{ProcessRemoteLinkTest.RemoteUseStaleLink}} and
> {{ProcessRemoteLinkTest.RemoteStaleLinkRelink}} are failing occasionally with
> the error:
> {code}
> [ RUN ] ProcessRemoteLinkTest.RemoteStaleLinkRelink
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0630 07:42:34.661110 1 process.cpp:1066] libprocess is initialized on
> 172.17.0.2:56294 with 16 worker threads
> E0630 07:42:34.666393 18765 process.cpp:2104] Failed to shutdown socket with
> fd 7: Transport endpoint is not connected
> /mesos/3rdparty/libprocess/src/tests/process_tests.cpp:1059: Failure
> Value of: exitedPid.isPending()
> Actual: false
> Expected: true
> [ FAILED ] ProcessRemoteLinkTest.RemoteStaleLinkRelink (56 ms)
> {code}
> There appears to be a race between establishing a socket connection and the
> test calling {{::shutdown}} on the socket. Under some circumstances, the
> {{::shutdown}} may actually result in failing the future in
> {{SocketManager::link_connect}} error and thereby trigger
> {{SocketManager::close}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5761) Filter orphaned task from /state.
[ https://issues.apache.org/jira/browse/MESOS-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358496#comment-15358496 ] Joerg Schad commented on MESOS-5761: The problem with orphaned tasks is that we don't necessarily access to the FrameworkInfo and hence we cannot authorize them as we would do for running/staged task (i.e., first check Task/TaskInfo and as fallback check FrameworInfo). I see three options for dealing with this problem: 1. Coarse granular authz for orphaned tasks Introduce a new acls/authz action for allowing a user to view orphaned tasks (i.e., all or none). 2. Add FrameworkInfo to Task (and then authorize orphaned Tasks the same way we would running/staged ones) The drawback here would be the increased size (and hence memory usage) for storing Tasks. FrameworkInfo can have large fields. Note that FrameworkInfo will be stored for each Task and hence potentially duplicated. 3. Add a new datastructure to the master storing a mapping of frameworkID to frameworkInfo (the frameworkInfo is part of Task and hence we can authorize orphaned Tasks the same way we would running/staged ones) This would lead to less storage overhead compared to 2, but might be a little more complex to implement as we have to fill/cleanup such datastructure. > Filter orphaned task from /state. > - > > Key: MESOS-5761 > URL: https://issues.apache.org/jira/browse/MESOS-5761 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad >Assignee: Joerg Schad > > Currently, we don't filter orphaned tasks based on user authorization. This > can potentially expose sensitive information to users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5761) Filter orphaned task from /state.
Joerg Schad created MESOS-5761: -- Summary: Filter orphaned task from /state. Key: MESOS-5761 URL: https://issues.apache.org/jira/browse/MESOS-5761 Project: Mesos Issue Type: Bug Reporter: Joerg Schad Assignee: Joerg Schad Currently, we don't filter orphaned tasks based on user authorization. This can potentially expose sensitive information to users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5709: --- Shepherd: Vinod Kone (was: Adam B) > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Blocker > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357722#comment-15357722 ] Joerg Schad commented on MESOS-5709: (note that the regenaration of the endpoint docu is missing, but seemingly there is supposed to be one big regeneration run before cutting the rc) > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Blocker > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355062#comment-15355062 ] Joerg Schad commented on MESOS-5709: Introduced authorization based filtering for /roles. https://reviews.apache.org/r/49369 Updateted documentation for roles endpoint filtering https://reviews.apache.org/r/49370/ > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Minor > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5711) Update AUTHORIZATION strings in endpoint help
[ https://issues.apache.org/jira/browse/MESOS-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5711: --- Shepherd: Alexander Rukletsov > Update AUTHORIZATION strings in endpoint help > - > > Key: MESOS-5711 > URL: https://issues.apache.org/jira/browse/MESOS-5711 > Project: Mesos > Issue Type: Task > Components: documentation, security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Minor > Labels: documentation, mesosphere, security > Fix For: 1.0.0 > > > The endpoint help macros support AUTHENTICATION and AUTHORIZATION sections. > We added AUTHORIZATION help for some of the newer endpoints, but not the > previously authenticated endpoints. > Authorization endpoints needing help string updates: > Master::Http::CREATE_VOLUMES_HELP > Master::Http::DESTROY_VOLUMES_HELP > Master::Http::RESERVE_HELP > Master::Http::STATE_HELP > Master::Http::STATESUMMARY_HELP > Master::Http::TEARDOWN_HELP > Master::Http::TASKS_HELP > Master::Http::UNRESERVE_HELP > Slave::Http::STATE_HELP -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5730) Sandbox access authorization should fail for non existing sandboxes.
[
https://issues.apache.org/jira/browse/MESOS-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353089#comment-15353089
]
Joerg Schad commented on MESOS-5730:
Proposed doc fix: https://reviews.apache.org/r/49319/
> Sandbox access authorization should fail for non existing sandboxes.
>
>
> Key: MESOS-5730
> URL: https://issues.apache.org/jira/browse/MESOS-5730
> Project: Mesos
> Issue Type: Bug
> Components: security
>Affects Versions: 1.0.0
>Reporter: Till Toenshoff
>Priority: Blocker
> Labels: authorization, mesosphere, security
> Fix For: 1.0.0
>
>
> The local authorizer currently tries to authorize {{ACCESS_SANDBOX}} even if
> no further object specification - e.g. {{framework_info}} or
> {{executor_info}}) where specified / available at that time.
> Given that there is likely no sandbox available if there was no
> {{executor_info}} provided, I think we should actually fail instead of allow
> or deny (403).
> A failure would result into an IMHO more appropriate ServiceUnavailable
> (503).
> See
> https://github.com/apache/mesos/commit/c8d67590064e35566274116cede9c6a733187b48#diff-dd692b1640b2628014feca01a94ba1e1R241
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5730) Sandbox access authorization should fail for non existing sandboxes.
[
https://issues.apache.org/jira/browse/MESOS-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353086#comment-15353086
]
Joerg Schad commented on MESOS-5730:
Short related issue:
The comment in authorizer.proto for `ACCESS_SANDBOX` is incorrect in that it
states that ` // This action will have a `FrameworkInfo` and `ExecutorInfo`
set,`. We either should fix the semantic (i.e., making them really required) or
fix the comment.
> Sandbox access authorization should fail for non existing sandboxes.
>
>
> Key: MESOS-5730
> URL: https://issues.apache.org/jira/browse/MESOS-5730
> Project: Mesos
> Issue Type: Bug
> Components: security
>Affects Versions: 1.0.0
>Reporter: Till Toenshoff
>Priority: Blocker
> Labels: authorization, mesosphere, security
> Fix For: 1.0.0
>
>
> The local authorizer currently tries to authorize {{ACCESS_SANDBOX}} even if
> no further object specification - e.g. {{framework_info}} or
> {{executor_info}}) where specified / available at that time.
> Given that there is likely no sandbox available if there was no
> {{executor_info}} provided, I think we should actually fail instead of allow
> or deny (403).
> A failure would result into an IMHO more appropriate ServiceUnavailable
> (503).
> See
> https://github.com/apache/mesos/commit/c8d67590064e35566274116cede9c6a733187b48#diff-dd692b1640b2628014feca01a94ba1e1R241
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352968#comment-15352968 ] Joerg Schad commented on MESOS-5709: Proposal: add VIEW_ROLE and adapt GET_WEIGHTS to use that as well. > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Minor > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5709) Authorization for /roles
[ https://issues.apache.org/jira/browse/MESOS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad reassigned MESOS-5709: -- Assignee: Joerg Schad (was: zhou xing) > Authorization for /roles > > > Key: MESOS-5709 > URL: https://issues.apache.org/jira/browse/MESOS-5709 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Minor > Labels: mesosphere, security > Fix For: 1.0.0 > > > The /roles endpoint exposes the list of all roles and their weights, as well > as the list of all frameworkIds registered with each role. This is a superset > of the information exposed on GET /weights, which we already protect. We > should protect the data in /roles the same way. > - Should we reuse VIEW_FRAMEWORK with role (from /state)? > - Should we add a new VIEW_ROLE and adapt GET_WEIGHTS to use it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5711) Update AUTHORIZATION strings in endpoint help
[ https://issues.apache.org/jira/browse/MESOS-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad reassigned MESOS-5711: -- Assignee: Joerg Schad > Update AUTHORIZATION strings in endpoint help > - > > Key: MESOS-5711 > URL: https://issues.apache.org/jira/browse/MESOS-5711 > Project: Mesos > Issue Type: Task > Components: documentation, security >Reporter: Adam B >Assignee: Joerg Schad >Priority: Minor > Labels: documentation, mesosphere, security > Fix For: 1.0.0 > > > The endpoint help macros support AUTHENTICATION and AUTHORIZATION sections. > We added AUTHORIZATION help for some of the newer endpoints, but not the > previously authenticated endpoints. > Authorization endpoints needing help string updates: > Master::Http::CREATE_VOLUMES_HELP > Master::Http::DESTROY_VOLUMES_HELP > Master::Http::RESERVE_HELP > Master::Http::STATE_HELP > Master::Http::STATESUMMARY_HELP > Master::Http::TEARDOWN_HELP > Master::Http::TASKS_HELP > Master::Http::UNRESERVE_HELP > Slave::Http::STATE_HELP -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5629) Agent segfaults after request to '/files/browse'
[
https://issues.apache.org/jira/browse/MESOS-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336016#comment-15336016
]
Joerg Schad commented on MESOS-5629:
https://reviews.apache.org/r/48849/
> Agent segfaults after request to '/files/browse'
>
>
> Key: MESOS-5629
> URL: https://issues.apache.org/jira/browse/MESOS-5629
> Project: Mesos
> Issue Type: Bug
> Environment: CentOS 7, Mesos 1.0.0-rc1 with patches
>Reporter: Greg Mann
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, security
> Fix For: 1.0.0
>
> Attachments: test-browse.py
>
>
> We observed a number of agent segfaults today on an internal testing cluster.
> Here is a log excerpt:
> {code}
> Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.522925 24830
> status_update_manager.cpp:392] Received status update acknowledgement (UUID:
> e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
> datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-
> Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.523006 24830
> status_update_manager.cpp:824] Checkpointing ACK for status update
> TASK_RUNNING (UUID: e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
> datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:29.147181 24824
> http.cpp:192] HTTP GET for /slave(1)/state from 10.10.0.87:33356
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** Aborted at 1466097149
> (unix time) try "date -d @1466097149" if you are using GNU date ***
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: PC: @ 0x7ff4d68b12a3
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** SIGSEGV (@0x0) received
> by PID 24818 (TID 0x7ff4d31ab700) from PID 0; stack trace: ***
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6431100
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d68b12a3
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7eced33
> process::dispatch<>()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7e7aad7
> _ZNSt17_Function_handlerIFN7process6FutureIbEERK6OptionISsEEZN5mesos8internal5slave9Framework15recoverExecutorERKNSA_5state13ExecutorStateEEUlS6_E_E9_M_invokeERKSt9_Any_dataS6_
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1752
> mesos::internal::FilesProcess::authorize()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1bea
> mesos::internal::FilesProcess::browse()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd6e43
> std::_Function_handler<>::_M_invoke()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d85478cb
> _ZZZN7process11ProcessBase5visitERKNS_9HttpEventEENKUlRKNS_6FutureI6OptionINS_4http14authentication20AuthenticationResultE0_clESC_ENKUlRKNS4_IbEEE1_clESG_
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551341
> process::ProcessManager::resume()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551647
> _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUt_vEEE6_M_runEv
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6909220
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6429dc5
> start_thread
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d615728d __clone
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service: main
> process exited, code=killed, status=11/SEGV
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: Unit dcos-mesos-slave.service
> entered failed state.
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service failed.
> Jun 16 17:12:34 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service holdoff
> time over, scheduling restart.
> {code}
> In every case, the stack trace indicates one of the {{/files/*}} endpoints; I
> observed this a number of times coming from {{browse()}}, and twice from
> {{read()}}.
> The agent was built from the 1.0.0-rc1 branch, with two cherry-picks applied:
> [this|https://reviews.apache.org/r/48563/] and
> [this|https://reviews.apache.org/r/48566/], which were done to repair a
> different [segfault issue|https://issues.apache.org/jira/browse/MESOS-5587]
> on the master and agent.
> Thanks go to [~bmahler] for digging into this a bit and discovering a
> possible cause
> [here|https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L5737-L5745],
> where use of {{defer()}} may be necessary to keep execution in the correct
> context.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (MESOS-5629) Agent segfaults after request to '/files/browse'
[
https://issues.apache.org/jira/browse/MESOS-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335967#comment-15335967
]
Joerg Schad commented on MESOS-5629:
Most likely hypotheses the issue is capturing `this` in
Framework::launchExecutor and Framework::recoverExecutor.
The `Framework` will out of scope, but the `this` pointer to this is still kept
in the lambda and hence dangling.
Our solution is to remove the `this` capture and replace it by a value copy of
the slave pid (which is the only attribute used from captured `this`).
We have not been able to reproduce this on an AWS instance. [~greggomann] could
you help out with verifying the patch (see next comment)?
> Agent segfaults after request to '/files/browse'
>
>
> Key: MESOS-5629
> URL: https://issues.apache.org/jira/browse/MESOS-5629
> Project: Mesos
> Issue Type: Bug
> Environment: CentOS 7, Mesos 1.0.0-rc1 with patches
>Reporter: Greg Mann
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, security
> Fix For: 1.0.0
>
> Attachments: test-browse.py
>
>
> We observed a number of agent segfaults today on an internal testing cluster.
> Here is a log excerpt:
> {code}
> Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.522925 24830
> status_update_manager.cpp:392] Received status update acknowledgement (UUID:
> e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
> datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-
> Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.523006 24830
> status_update_manager.cpp:824] Checkpointing ACK for status update
> TASK_RUNNING (UUID: e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
> datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:29.147181 24824
> http.cpp:192] HTTP GET for /slave(1)/state from 10.10.0.87:33356
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** Aborted at 1466097149
> (unix time) try "date -d @1466097149" if you are using GNU date ***
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: PC: @ 0x7ff4d68b12a3
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** SIGSEGV (@0x0) received
> by PID 24818 (TID 0x7ff4d31ab700) from PID 0; stack trace: ***
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6431100
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d68b12a3
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7eced33
> process::dispatch<>()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7e7aad7
> _ZNSt17_Function_handlerIFN7process6FutureIbEERK6OptionISsEEZN5mesos8internal5slave9Framework15recoverExecutorERKNSA_5state13ExecutorStateEEUlS6_E_E9_M_invokeERKSt9_Any_dataS6_
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1752
> mesos::internal::FilesProcess::authorize()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1bea
> mesos::internal::FilesProcess::browse()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd6e43
> std::_Function_handler<>::_M_invoke()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d85478cb
> _ZZZN7process11ProcessBase5visitERKNS_9HttpEventEENKUlRKNS_6FutureI6OptionINS_4http14authentication20AuthenticationResultE0_clESC_ENKUlRKNS4_IbEEE1_clESG_
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551341
> process::ProcessManager::resume()
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551647
> _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUt_vEEE6_M_runEv
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6909220
> (unknown)
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6429dc5
> start_thread
> Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d615728d __clone
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service: main
> process exited, code=killed, status=11/SEGV
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: Unit dcos-mesos-slave.service
> entered failed state.
> Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service failed.
> Jun 16 17:12:34 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service holdoff
> time over, scheduling restart.
> {code}
> In every case, the stack trace indicates one of the {{/files/*}} endpoints; I
> observed this a number of times coming from {{browse()}}, and twice from
> {{read()}}.
> The agent was built from the 1.0.0-rc1 branch, with two cherry-picks applied:
> [this|https://reviews.apache.org/r/48563/] and
>
[jira] [Comment Edited] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335528#comment-15335528
]
Joerg Schad edited comment on MESOS-5588 at 6/17/16 6:37 AM:
-
1) Yes that happens to every protobuf conversion (see my early comments on this
ticket about changing the parsing). But in this case it yields a security
critical issue. IMO this is part of the scope of this ticket (Improve error
handling when parsing acls, but if you want to split the 1.0 blocker relevant
part into an extra ticket that seems fine with me.
I agree that the second part is not a blocker (as it does not involve an API
change), but I would not say that it is a low priority wish
was (Author: js84):
I agree that the second part is not a blocker (as it does not involve an API
change), but I would not say that it is a low priority wish
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335528#comment-15335528
]
Joerg Schad commented on MESOS-5588:
I agree that the second part is not a blocker (as it does not involve an API
change), but I would not say that it is a low priority wish
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5629) Agent segfaults after request to '/files/browse'
[
https://issues.apache.org/jira/browse/MESOS-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-5629:
---
Description:
We observed a number of agent segfaults today on an internal testing cluster.
Here is a log excerpt:
{code}
Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.522925 24830
status_update_manager.cpp:392] Received status update acknowledgement (UUID:
e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
6d4248cd-2832-4152-b5d0-defbf36f6759-
Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.523006 24830
status_update_manager.cpp:824] Checkpointing ACK for status update TASK_RUNNING
(UUID: e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
6d4248cd-2832-4152-b5d0-defbf36f6759-
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:29.147181 24824
http.cpp:192] HTTP GET for /slave(1)/state from 10.10.0.87:33356
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** Aborted at 1466097149
(unix time) try "date -d @1466097149" if you are using GNU date ***
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: PC: @ 0x7ff4d68b12a3
(unknown)
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: *** SIGSEGV (@0x0) received
by PID 24818 (TID 0x7ff4d31ab700) from PID 0; stack trace: ***
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6431100 (unknown)
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d68b12a3 (unknown)
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7eced33
process::dispatch<>()
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7e7aad7
_ZNSt17_Function_handlerIFN7process6FutureIbEERK6OptionISsEEZN5mesos8internal5slave9Framework15recoverExecutorERKNSA_5state13ExecutorStateEEUlS6_E_E9_M_invokeERKSt9_Any_dataS6_
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1752
mesos::internal::FilesProcess::authorize()
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd1bea
mesos::internal::FilesProcess::browse()
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d7bd6e43
std::_Function_handler<>::_M_invoke()
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d85478cb
_ZZZN7process11ProcessBase5visitERKNS_9HttpEventEENKUlRKNS_6FutureI6OptionINS_4http14authentication20AuthenticationResultE0_clESC_ENKUlRKNS4_IbEEE1_clESG_
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551341
process::ProcessManager::resume()
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d8551647
_ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUt_vEEE6_M_runEv
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6909220 (unknown)
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d6429dc5
start_thread
Jun 16 17:12:29 ip-10-10-0-87 mesos-slave[24818]: @ 0x7ff4d615728d __clone
Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service: main
process exited, code=killed, status=11/SEGV
Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: Unit dcos-mesos-slave.service entered
failed state.
Jun 16 17:12:29 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service failed.
Jun 16 17:12:34 ip-10-10-0-87 systemd[1]: dcos-mesos-slave.service holdoff time
over, scheduling restart.
{code}
In every case, the stack trace indicates one of the {{/files/*}} endpoints; I
observed this a number of times coming from {{browse()}}, and twice from
{{read()}}.
The agent was built from the 1.0.0-rc1 branch, with two cherry-picks applied:
[this|https://reviews.apache.org/r/48563/] and
[this|https://reviews.apache.org/r/48566/], which were done to repair a
different [segfault issue|https://issues.apache.org/jira/browse/MESOS-5587] on
the master and agent.
Thanks go to [~bmahler] for digging into this a bit and discovering a possible
cause
[here|https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L5737-L5745],
where use of {{defer()}} may be necessary to keep execution in the correct
context.
was:
We observed a number of agent segfaults today on an internal testing cluster.
Here is a log excerpt:
{code}
Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.522925 24830
status_update_manager.cpp:392] Received status update acknowledgement (UUID:
e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
6d4248cd-2832-4152-b5d0-defbf36f6759-
Jun 16 17:12:28 ip-10-10-0-87 mesos-slave[24818]: I0616 17:12:28.523006 24830
status_update_manager.cpp:824] Checkpointing ACK for status update TASK_RUNNING
(UUID: e79ab0f4-2fa2-4df2-9b59-89b97a482167) for task
datadog-monitor.804b138b-33e5-11e6-ac16-566ccbdde23e of framework
6d4248cd-2832-4152-b5d0-defbf36f6759-
Jun 16 17:12:29 ip-10-10-0-87
[jira] [Commented] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334888#comment-15334888
]
Joerg Schad commented on MESOS-5588:
The problem is that those fields are optional, ie., we get a valid protobuf
which passes validation.
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5615) When using command executor, the ExecutorInfo is useless for sandbox authorization
[
https://issues.apache.org/jira/browse/MESOS-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334830#comment-15334830
]
Joerg Schad commented on MESOS-5615:
A few more comments on this: IMO a general issue of this creating an
executorInfo on the agent (i.e., already before this the fixes proposed here)
is that the agent/state and master/state will differ as the agent contains this
new executorInfo.
Especially this means that the copied Labels/DiscoveryInfo can appear both on
the TaskInfo and the ExecutorInfo. These fields can be custom generated by
frameworks and custom consumed by external tool. So we should make sure users
(both framework writers and operators/tool writers) are aware of this.
> When using command executor, the ExecutorInfo is useless for sandbox
> authorization
> --
>
> Key: MESOS-5615
> URL: https://issues.apache.org/jira/browse/MESOS-5615
> Project: Mesos
> Issue Type: Bug
> Components: modules, security, slave
>Affects Versions: 1.0.0
>Reporter: Alexander Rojas
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, modularization, security
> Fix For: 1.0.0
>
>
> The design for sandbox access authorization uses the {{ExecutorInfo}}
> associated with the task as the main authorization space and the
> {{FrameworkInfo}} as a secondary one. This allows module writes to use fields
> such a labels for authorization.
> When a task uses the _command executor_ it doesn't provide an
> {{ExecutorInfo}}, but the info object is generated automatically inside the
> agent. As such, information which could be used for authorization (e.g.
> labels) is not available for authorization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334813#comment-15334813
]
Joerg Schad edited comment on MESOS-5588 at 6/16/16 10:17 PM:
--
The above review (if all issues are resolved) solves only one part of the
problem: i.e., that the object is missing.
This is imo the most prominent problem as it might end up allowing some action
for ANY object.
But the same problem can appear on the layer above, i.e., typos on the action
name level:
{code}
"view_frameworks": [
{
"principals": { "type": "ANY" },
"user": { "type": "NONE" }
}
]
{code}
This would result in the ACLs for that action are not being considered.
One potential way to check that all acls are parsed could be the following
(note that a problem here is that we validate the protobuf, but the protobuf is
a valid):
We could get the action count from the json file (count the objects) and
compare it to the action count in the protobuf.
Any other ideas?
was (Author: js84):
The above review (if all issues are resolved) solves only one part of the
problem: i.e., that the object is missing.
This is imo the most prominent problem as it might end up allowing some action
for ANY object.
But the same problem can appear on the layer above, i.e., typos on the action
name level:
{code}
"view_frameworks": [
{
"principals": { "type": "ANY" },
"usr": { "type": "NONE" }
}
]
{code}
This would result in the ACLs for that action are not being considered.
One potential way to check that all acls are parsed could be the following
(note that a problem here is that we validate the protobuf, but the protobuf is
a valid):
We could get the action count from the json file (count the objects) and
compare it to the action count in the protobuf.
Any other ideas?
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334813#comment-15334813
]
Joerg Schad commented on MESOS-5588:
The above review (if all issues are resolved) solves only one part of the
problem: i.e., that the object is missing.
This is imo the most prominent problem as it might end up allowing some action
for ANY object.
But the same problem can appear on the layer above, i.e., typos on the action
name level:
{code}
"view_frameworks": [
{
"principals": { "type": "ANY" },
"usr": { "type": "NONE" }
}
]
{code}
This would result in the ACLs for that action are not being considered.
One potential way to check that all acls are parsed could be the following
(note that a problem here is that we validate the protobuf, but the protobuf is
a valid):
We could get the action count from the json file (count the objects) and
compare it to the action count in the protobuf.
Any other ideas?
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5605) Improve documentation for using persistent volumes.
[ https://issues.apache.org/jira/browse/MESOS-5605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5605: --- Assignee: (was: Joerg Schad) > Improve documentation for using persistent volumes. > > > Key: MESOS-5605 > URL: https://issues.apache.org/jira/browse/MESOS-5605 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad > > When using persistent volumes at a arangoDB we ran into a few pitfalls. > We should document them in order for others to avoid those issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5615) When using command executor, the ExecutorInfo is useless for sandbox authorization
[
https://issues.apache.org/jira/browse/MESOS-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332751#comment-15332751
]
Joerg Schad edited comment on MESOS-5615 at 6/16/16 12:58 PM:
--
Refactored sandbox authorization logic to use ObjectAuthorizer.
https://reviews.apache.org/r/48764/
Added 'labels' and 'discovery' to generated 'ExecutorInfo'.
https://reviews.apache.org/r/48765/
Added tests for sandbox authorization.
Review: https://reviews.apache.org/r/48789
Added note about generation of `ExecutorInfo` for `CommandInfo`.
Review: https://reviews.apache.org/r/48790
was (Author: js84):
Refactored sandbox authorization logic to use ObjectAuthorizer.
https://reviews.apache.org/r/48764/
Added 'labels' and 'discovery' to generated 'ExecutorInfo'.
https://reviews.apache.org/r/48765/
> When using command executor, the ExecutorInfo is useless for sandbox
> authorization
> --
>
> Key: MESOS-5615
> URL: https://issues.apache.org/jira/browse/MESOS-5615
> Project: Mesos
> Issue Type: Bug
> Components: modules, security, slave
>Affects Versions: 1.0.0
>Reporter: Alexander Rojas
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, modularization, security
> Fix For: 1.0.0
>
>
> The design for sandbox access authorization uses the {{ExecutorInfo}}
> associated with the task as the main authorization space and the
> {{FrameworkInfo}} as a secondary one. This allows module writes to use fields
> such a labels for authorization.
> When a task uses the _command executor_ it doesn't provide an
> {{ExecutorInfo}}, but the info object is generated automatically inside the
> agent. As such, information which could be used for authorization (e.g.
> labels) is not available for authorization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1540#comment-1540
]
Joerg Schad commented on MESOS-5588:
One weird inconsistency is that we have different optional/required semantics
in different acls:
{code}
// Specifies which roles a principal can reserve resources for.
message ReserveResources {
// Subjects: Framework principal or Operator username.
required Entity principals = 1;
// Objects: The principal(s) can reserve resources for these roles.
required Entity roles = 2;
}
{code}
vs.
{code}
// Which principals are authorized to access the Mesos logs.
message AccessMesosLog {
// Subjects: HTTP Username.
required Entity principals = 1;
// Objects: Given implicitly. Use Entity type ANY or NONE to allow or deny
// access.
optional Entity logs = 2;
}
{code}
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5615) When using command executor, the ExecutorInfo is useless for sandbox authorization
[
https://issues.apache.org/jira/browse/MESOS-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332751#comment-15332751
]
Joerg Schad commented on MESOS-5615:
Refactored sandbox authorization logic to use ObjectAuthorizer.
https://reviews.apache.org/r/48764/
Added 'labels' and 'discovery' to generated 'ExecutorInfo'.
https://reviews.apache.org/r/48765/
> When using command executor, the ExecutorInfo is useless for sandbox
> authorization
> --
>
> Key: MESOS-5615
> URL: https://issues.apache.org/jira/browse/MESOS-5615
> Project: Mesos
> Issue Type: Bug
> Components: modules, security, slave
>Affects Versions: 1.0.0
>Reporter: Alexander Rojas
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, modularization, security
> Fix For: 1.0.0
>
>
> The design for sandbox access authorization uses the {{ExecutorInfo}}
> associated with the task as the main authorization space and the
> {{FrameworkInfo}} as a secondary one. This allows module writes to use fields
> such a labels for authorization.
> When a task uses the _command executor_ it doesn't provide an
> {{ExecutorInfo}}, but the info object is generated automatically inside the
> agent. As such, information which could be used for authorization (e.g.
> labels) is not available for authorization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-5615) When using command executor, the ExecutorInfo is useless for sandbox authorization
[
https://issues.apache.org/jira/browse/MESOS-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad reassigned MESOS-5615:
--
Assignee: Joerg Schad (was: Alexander Rojas)
> When using command executor, the ExecutorInfo is useless for sandbox
> authorization
> --
>
> Key: MESOS-5615
> URL: https://issues.apache.org/jira/browse/MESOS-5615
> Project: Mesos
> Issue Type: Bug
> Components: modules, security, slave
>Affects Versions: 1.0.0
>Reporter: Alexander Rojas
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authorization, mesosphere, modularization, security
> Fix For: 1.0.0
>
>
> The design for sandbox access authorization uses the {{ExecutorInfo}}
> associated with the task as the main authorization space and the
> {{FrameworkInfo}} as a secondary one. This allows module writes to use fields
> such a labels for authorization.
> When a task uses the _command executor_ it doesn't provide an
> {{ExecutorInfo}}, but the info object is generated automatically inside the
> agent. As such, information which could be used for authorization (e.g.
> labels) is not available for authorization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5583) Improve authorization documentation when setting permissive flag.
[ https://issues.apache.org/jira/browse/MESOS-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5583: --- Shepherd: Till Toenshoff (was: Adam B) > Improve authorization documentation when setting permissive flag. > - > > Key: MESOS-5583 > URL: https://issues.apache.org/jira/browse/MESOS-5583 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > > A common problem for a users starting to use acls is that once they set > `permisse = false` and not add acls allowing common operations (e.g., > register_framework) their Mesos cluster don't > behave as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5604) Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`).
[ https://issues.apache.org/jira/browse/MESOS-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326479#comment-15326479 ] Joerg Schad commented on MESOS-5604: Fixed continuation logic in docker.cpp. https://reviews.apache.org/r/48599/ > Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`). > --- > > Key: MESOS-5604 > URL: https://issues.apache.org/jira/browse/MESOS-5604 > Project: Mesos > Issue Type: Task >Reporter: Joerg Schad >Assignee: Joerg Schad > > We recently experienced a race condition (MESOS-5587) as we used .then() > instead of .then(defer(self() ). > When looking at the code base we found a number of other potentially wrong > uses which could cause race conditions. > This Jira is supposed to track the investigation effort in this matter. > Note that this problem does not only apply to .then, but also for example to > .onAny(). From looking at the codebase, it just seems we are more disciplined > when using .onAny(). Hence we first look at .then(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5605) Improve documentation for using persistent volumes.
Joerg Schad created MESOS-5605: -- Summary: Improve documentation for using persistent volumes. Key: MESOS-5605 URL: https://issues.apache.org/jira/browse/MESOS-5605 Project: Mesos Issue Type: Documentation Reporter: Joerg Schad Assignee: Joerg Schad When using persistent volumes at a arangoDB we ran into a few pitfalls. We should document them in order for others to avoid those issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5604) Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`).
[ https://issues.apache.org/jira/browse/MESOS-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5604: --- Description: We recently experienced a race condition (MESOS-5587) as we used .then() instead of .then(defer(self() ). When looking at the code base we found a number of other potentially wrong uses which could cause race conditions. This Jira is supposed to track the investigation effort in this matter. Note that this problem does not only apply to .then, but also for example to .onAny(). From looking at the codebase, it just seems we are more disciplined when using .onAny(). Hence we first look at .then(). was: We recently experienced a race condition (MESOS-5587) as we used .then() instead of .then(defer(self() ). When looking at the code base we found a number of other potentially wrong uses which could cause race conditions. This Jira is supposed to track the investigation effort in this matter. > Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`). > --- > > Key: MESOS-5604 > URL: https://issues.apache.org/jira/browse/MESOS-5604 > Project: Mesos > Issue Type: Task >Reporter: Joerg Schad >Assignee: Joerg Schad > > We recently experienced a race condition (MESOS-5587) as we used .then() > instead of .then(defer(self() ). > When looking at the code base we found a number of other potentially wrong > uses which could cause race conditions. > This Jira is supposed to track the investigation effort in this matter. > Note that this problem does not only apply to .then, but also for example to > .onAny(). From looking at the codebase, it just seems we are more disciplined > when using .onAny(). Hence we first look at .then(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5604) Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`).
[ https://issues.apache.org/jira/browse/MESOS-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326035#comment-15326035 ] Joerg Schad commented on MESOS-5604: Started document collecting potential misuses of .then. Please note this is WIP! https://docs.google.com/document/d/1jCe-3wAgPMIn-m_OjlNckRVa4LOxHWgr7WHJnuFseeQ > Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`). > --- > > Key: MESOS-5604 > URL: https://issues.apache.org/jira/browse/MESOS-5604 > Project: Mesos > Issue Type: Task >Reporter: Joerg Schad >Assignee: Joerg Schad > > We recently experienced a race condition (MESOS-5587) as we used .then() > instead of .then(defer(self() ). > When looking at the code base we found a number of other potentially wrong > uses which could cause race conditions. > This Jira is supposed to track the investigation effort in this matter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5604) Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`).
Joerg Schad created MESOS-5604: -- Summary: Check for incorrect use of `.then` (as opposed to `.then(defer(self() ))`). Key: MESOS-5604 URL: https://issues.apache.org/jira/browse/MESOS-5604 Project: Mesos Issue Type: Task Reporter: Joerg Schad Assignee: Joerg Schad We recently experienced a race condition (MESOS-5587) as we used .then() instead of .then(defer(self() ). When looking at the code base we found a number of other potentially wrong uses which could cause race conditions. This Jira is supposed to track the investigation effort in this matter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5587) FullFrameworkWriter makes master segmentation fault.
[
https://issues.apache.org/jira/browse/MESOS-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325273#comment-15325273
]
Joerg Schad commented on MESOS-5587:
Potentially the sandbox authorization has the same issue:
see https://reviews.apache.org/r/48566/
> FullFrameworkWriter makes master segmentation fault.
>
>
> Key: MESOS-5587
> URL: https://issues.apache.org/jira/browse/MESOS-5587
> Project: Mesos
> Issue Type: Bug
>Reporter: Gilbert Song
>Assignee: Joerg Schad
>Priority: Blocker
> Labels: authentication, mesosphere
> Fix For: 1.0.0
>
>
> FullFrameworkWriter::operator() may take down the master. Here is the log:
> {noformat}
> Jun 09 02:28:42 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:42.147253
> 18633 master.cpp:5772] Sending 1 offers to framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-0001 (chronos) at
> scheduler-c9cb7c2c-ae6b-4a34-8663-6a52980161c1@10.10.0.20:39285
> Jun 09 02:28:42 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:42.148890
> 18637 master.cpp:4066] Processing DECLINE call for offers: [
> 7567c338-3ae5-4a84-bf5b-6a75a8a49341-O992 ] for framework
> 6d4248cd-2832-4152-b5d0-defbf36f6759-0001 (chronos) at
> scheduler-c9cb7c2c-ae6b-4a34-8663-6a52980161c1@10.10.0.20:39285
> Jun 09 02:28:42 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:42.639813
> 18632 http.cpp:483] HTTP GET for /master/state-summary from 10.10.0.180:45790
> with User-Agent='python-requests/2.6.0 CPython/3.4.2
> Linux/3.10.0-327.10.1.el7.x86_64'
> Jun 09 02:28:42 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:42.890702
> 18632 http.cpp:483] HTTP GET for /master/state from 10.10.0.181:33830 with
> User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36'
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.139240
> 18639 http.cpp:483] HTTP GET for /master/state-summary from 10.10.0.181:33831
> with User-Agent='python-requests/2.6.0 CPython/3.4.2
> Linux/3.10.0-327.18.2.el7.x86_64'
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.148582
> 18633 master.cpp:5772] Sending 1 offers to framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 (Long Lived Framework (C++))
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.150388
> 18635 http.cpp:483] HTTP POST for /master/api/v1/scheduler from
> 10.10.0.178:51645
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.150645
> 18635 master.cpp:3457] Processing ACCEPT call for offers: [
> 7567c338-3ae5-4a84-bf5b-6a75a8a49341-O993 ] on agent
> 091e9c3f-8a01-4890-8790-48b75fd81b40-S0 at slave(1)@10.10.0.20:5051
> (10.10.0.20) for framework 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 (Long
> Lived Framework (C++))
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.151268
> 18635 master.hpp:178] Adding task 5699 with resources cpus(*):0.001; mem(*):1
> on agent 091e9c3f-8a01-4890-8790-48b75fd81b40-S0 (10.10.0.20)
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.151322
> 18635 master.cpp:3946] Launching task 5699 of framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 (Long Lived Framework (C++)) with
> resources cpus(*):0.001; mem(*):1 on agent
> 091e9c3f-8a01-4890-8790-48b75fd81b40-S0 at slave(1)@10.10.0.20:5051
> (10.10.0.20)
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.160475
> 18635 master.cpp:5211] Status update TASK_RUNNING (UUID:
> 3f651ba8-7c80-4ac0-ae18-579371ec82d5) for task 5699 of framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 from agent
> 091e9c3f-8a01-4890-8790-48b75fd81b40-S0 at slave(1)@10.10.0.20:5051
> (10.10.0.20)
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.160516
> 18635 master.cpp:5259] Forwarding status update TASK_RUNNING (UUID:
> 3f651ba8-7c80-4ac0-ae18-579371ec82d5) for task 5699 of framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.160645
> 18635 master.cpp:6871] Updating the state of task 5699 of framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 (latest state: TASK_RUNNING, status
> update state: TASK_RUNNING)
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.161842
> 18639 http.cpp:483] HTTP POST for /master/api/v1/scheduler from
> 10.10.0.178:51645
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.161912
> 18639 master.cpp:4365] Processing ACKNOWLEDGE call
> 3f651ba8-7c80-4ac0-ae18-579371ec82d5 for task 5699 of framework
> 4c6031e7-4cfd-4219-89b2-d19c7101e045-0001 (Long Lived Framework (C++)) on
> agent 091e9c3f-8a01-4890-8790-48b75fd81b40-S0
> Jun 09 02:28:43 ip-10-10-0-180 mesos-master[18627]: I0609 02:28:43.556354
>
[jira] [Commented] (MESOS-5588) Improve error handling when parsing acls.
[
https://issues.apache.org/jira/browse/MESOS-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324043#comment-15324043
]
Joerg Schad commented on MESOS-5588:
The problem arises from the fact that the protobuf parser ignores unknown field
values:
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/protobuf.hpp#L599
> Improve error handling when parsing acls.
> -
>
> Key: MESOS-5588
> URL: https://issues.apache.org/jira/browse/MESOS-5588
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>
> During parsing of the authorizer errors are ignored. This can lead to
> undetected security issues.
> Consider the following acl with an typo (usr instead of od user)
> {code}
>"view_frameworks": [
> {
> "principals": { "type": "ANY" },
> "usr": { "type": "NONE" }
> }
> ]
> {code}
> When the master is started with these flags it will interprete the acl int he
> following way which gives any principal access to any framework.
> {noformat}
> view_frameworks {
> principals {
> type: ANY
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-5588) Improve error handling when parsing acls.
Joerg Schad created MESOS-5588:
--
Summary: Improve error handling when parsing acls.
Key: MESOS-5588
URL: https://issues.apache.org/jira/browse/MESOS-5588
Project: Mesos
Issue Type: Improvement
Reporter: Joerg Schad
Assignee: Joerg Schad
During parsing of the authorizer errors are ignored. This can lead to
undetected security issues.
Consider the following acl with an typo (usr instead of od user)
{code}
"view_frameworks": [
{
"principals": { "type": "ANY" },
"usr": { "type": "NONE" }
}
]
{code}
When the master is started with these flags it will interprete the acl int he
following way which gives any principal access to any framework.
{noformat}
view_frameworks {
principals {
type: ANY
}
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5570) Improve CHANGELOG and upgrades.md
[ https://issues.apache.org/jira/browse/MESOS-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5570: --- Sprint: Mesosphere Sprint 37 > Improve CHANGELOG and upgrades.md > - > > Key: MESOS-5570 > URL: https://issues.apache.org/jira/browse/MESOS-5570 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > Fix For: 1.0.0 > > > Currently we have a lot of data duplication between the CHANGELOG and > upgrades.md. We should try to improve this and potentially make the CHANGLOG > a markdown file as well. For inspiration see the Hadoop changelog: > https://github.com/apache/hadoop/blob/2e1d0ff4e901b8313c8d71869735b94ed8bc40a0/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5583) Improve authorization documentation when setting permissive flag.
[ https://issues.apache.org/jira/browse/MESOS-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5583: --- Sprint: Mesosphere Sprint 37 > Improve authorization documentation when setting permissive flag. > - > > Key: MESOS-5583 > URL: https://issues.apache.org/jira/browse/MESOS-5583 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > > A common problem for a users starting to use acls is that once they set > `permisse = false` and not add acls allowing common operations (e.g., > register_framework) their Mesos cluster don't > behave as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5583) Improve authorization documentation when setting permissive flag.
[ https://issues.apache.org/jira/browse/MESOS-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5583: --- Shepherd: Adam B Description: A common problem for a users starting to use acls is that once they set `permisse = false` and not add acls allowing common operations (e.g., register_framework) their Mesos cluster don't behave as expected. was: A common problem for a users starting to use acls is that once they set `permisse = false` and not add acls allowing common operations (e.g., register_framework) their Mesos cluster don't behave as expected. > Improve authorization documentation when setting permissive flag. > - > > Key: MESOS-5583 > URL: https://issues.apache.org/jira/browse/MESOS-5583 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > > A common problem for a users starting to use acls is that once they set > `permisse = false` and not add acls allowing common operations (e.g., > register_framework) their Mesos cluster don't > behave as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5583) Improve authorization documentation when setting permissive flag.
[ https://issues.apache.org/jira/browse/MESOS-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5583: --- Description: A common problem for a users starting to use acls is that once they set `permisse = false` and not add acls allowing common operations (e.g., register_framework) their Mesos cluster don't behave as expected. was: A common problem for a users starting to use acls is that once they set `permisse = false` and not add acls allowing common operations (e.g., register_framework) their Mesos cluster don't behave as expected. > Improve authorization documentation when setting permissive flag. > - > > Key: MESOS-5583 > URL: https://issues.apache.org/jira/browse/MESOS-5583 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > > A common problem for a users starting to use acls is that once they set > `permisse = false` and > not add acls allowing common operations (e.g., register_framework) their > Mesos cluster don't > behave as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5583) Improve authorization documentation when setting permissive flag.
Joerg Schad created MESOS-5583: -- Summary: Improve authorization documentation when setting permissive flag. Key: MESOS-5583 URL: https://issues.apache.org/jira/browse/MESOS-5583 Project: Mesos Issue Type: Documentation Reporter: Joerg Schad Assignee: Joerg Schad A common problem for a users starting to use acls is that once they set `permisse = false` and not add acls allowing common operations (e.g., register_framework) their Mesos cluster don't behave as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5570) Improve CHANGELOG and upgrades.md
[ https://issues.apache.org/jira/browse/MESOS-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320602#comment-15320602 ] Joerg Schad edited comment on MESOS-5570 at 6/8/16 2:08 PM: Some prework Included missing changes for 1.0RC https://reviews.apache.org/r/48272/ Used tense consistenly in CHANGELOG https://reviews.apache.org/r/48271/ was (Author: js84): Included missing changes for 1.0RC https://reviews.apache.org/r/48272/ Used tense consistenly in CHANGELOG https://reviews.apache.org/r/48271/ > Improve CHANGELOG and upgrades.md > - > > Key: MESOS-5570 > URL: https://issues.apache.org/jira/browse/MESOS-5570 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > Fix For: 1.0.0 > > > Currently we have a lot of data duplication between the CHANGELOG and > upgrades.md. We should try to improve this and potentially make the CHANGLOG > a markdown file as well. For inspiration see the Hadoop changelog: > https://github.com/apache/hadoop/blob/2e1d0ff4e901b8313c8d71869735b94ed8bc40a0/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5570) Improve CHANGELOG and upgrades.md
[ https://issues.apache.org/jira/browse/MESOS-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320602#comment-15320602 ] Joerg Schad commented on MESOS-5570: Included missing changes for 1.0RC https://reviews.apache.org/r/48272/ Used tense consistenly in CHANGELOG https://reviews.apache.org/r/48271/ > Improve CHANGELOG and upgrades.md > - > > Key: MESOS-5570 > URL: https://issues.apache.org/jira/browse/MESOS-5570 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad > Fix For: 1.0.0 > > > Currently we have a lot of data duplication between the CHANGELOG and > upgrades.md. We should try to improve this and potentially make the CHANGLOG > a markdown file as well. For inspiration see the Hadoop changelog: > https://github.com/apache/hadoop/blob/2e1d0ff4e901b8313c8d71869735b94ed8bc40a0/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5570) Improve CHANGELOG and upgrades.md
Joerg Schad created MESOS-5570: -- Summary: Improve CHANGELOG and upgrades.md Key: MESOS-5570 URL: https://issues.apache.org/jira/browse/MESOS-5570 Project: Mesos Issue Type: Documentation Reporter: Joerg Schad Assignee: Joerg Schad Fix For: 1.0.0 Currently we have a lot of data duplication between the CHANGELOG and upgrades.md. We should try to improve this and potentially make the CHANGLOG a markdown file as well. For inspiration see the Hadoop changelog: https://github.com/apache/hadoop/blob/2e1d0ff4e901b8313c8d71869735b94ed8bc40a0/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5405) Make fields in authorization::Request protobuf optional.
[
https://issues.apache.org/jira/browse/MESOS-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316326#comment-15316326
]
Joerg Schad commented on MESOS-5405:
Added documentation and tests:
Fixed documentation for MESOS-5405.
https://reviews.apache.org/r/48263/
Added test for optional request.object field.
https://reviews.apache.org/r/48264/
> Make fields in authorization::Request protobuf optional.
>
>
> Key: MESOS-5405
> URL: https://issues.apache.org/jira/browse/MESOS-5405
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rukletsov
>Assignee: Till Toenshoff
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> Currently {{authorization::Request}} protobuf declares {{subject}} and
> {{object}} as required fields. However, in the codebase we not always set
> them, which renders the message in the uninitialized state, for example:
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/common/http.cpp#L603
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/master/http.cpp#L2057
> I believe that the reason why we don't see issues related to this is because
> we never send authz requests over the wire, i.e., never serialize/deserialize
> them. However, they are still invalid protobuf messages. Moreover, some
> external authorizers may serialize these messages.
> We can either ensure all required fields are set or make both {{subject}} and
> {{object}} fields optional. This will also require updating local authorizer,
> which should properly handle the situation when these fields are absent. We
> may also want to notify authors of external authorizers to update their code
> accordingly.
> It looks like no deprecation is necessary, mainly because we
> already—erroneously!—treat these fields as optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5405) Make fields in authorization::Request protobuf optional.
[
https://issues.apache.org/jira/browse/MESOS-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308861#comment-15308861
]
Joerg Schad commented on MESOS-5405:
https://reviews.apache.org/r/48101/
> Make fields in authorization::Request protobuf optional.
>
>
> Key: MESOS-5405
> URL: https://issues.apache.org/jira/browse/MESOS-5405
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rukletsov
>Assignee: Till Toenshoff
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> Currently {{authorization::Request}} protobuf declares {{subject}} and
> {{object}} as required fields. However, in the codebase we not always set
> them, which renders the message in the uninitialized state, for example:
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/common/http.cpp#L603
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/master/http.cpp#L2057
> I believe that the reason why we don't see issues related to this is because
> we never send authz requests over the wire, i.e., never serialize/deserialize
> them. However, they are still invalid protobuf messages. Moreover, some
> external authorizers may serialize these messages.
> We can either ensure all required fields are set or make both {{subject}} and
> {{object}} fields optional. This will also require updating local authorizer,
> which should properly handle the situation when these fields are absent. We
> may also want to notify authors of external authorizers to update their code
> accordingly.
> It looks like no deprecation is necessary, mainly because we
> already—erroneously!—treat these fields as optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5405) Make fields in authorization::Request protobuf optional.
[
https://issues.apache.org/jira/browse/MESOS-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308700#comment-15308700
]
Joerg Schad commented on MESOS-5405:
[~tillt] [~adam-mesos] [~mcypark]
This breaks some assumptions of the current `authorized` interface which assume
`subject` and `object` are set (see below).
In order to accomodate for this these new optional fields i would propose the
following
1. Change getObjectApprover's signatures to accept Option, Option
2. Change objectApprover->approved() signature to accept an Option
(and adapt the logic in approved for the LocalAuthorizerObjectApprover to deal
with the None -> Any conversion)
```
Future authorized(const authorization::Request& request)
{
return getObjectApprover(request.subject(), request.action())
.then([=](const Owned& objectApprover) -> Future {
ObjectApprover::Object object(request.object());
Try result = objectApprover->approved(object);
if (result.isError()) {
return Failure(result.error());
}
return result.get();
});
}
```
> Make fields in authorization::Request protobuf optional.
>
>
> Key: MESOS-5405
> URL: https://issues.apache.org/jira/browse/MESOS-5405
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rukletsov
>Assignee: Till Toenshoff
>Priority: Blocker
> Labels: mesosphere, security
> Fix For: 1.0.0
>
>
> Currently {{authorization::Request}} protobuf declares {{subject}} and
> {{object}} as required fields. However, in the codebase we not always set
> them, which renders the message in the uninitialized state, for example:
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/common/http.cpp#L603
> *
> https://github.com/apache/mesos/blob/0bfd6999ebb55ddd45e2c8566db17ab49bc1ffec/src/master/http.cpp#L2057
> I believe that the reason why we don't see issues related to this is because
> we never send authz requests over the wire, i.e., never serialize/deserialize
> them. However, they are still invalid protobuf messages. Moreover, some
> external authorizers may serialize these messages.
> We can either ensure all required fields are set or make both {{subject}} and
> {{object}} fields optional. This will also require updating local authorizer,
> which should properly handle the situation when these fields are absent. We
> may also want to notify authors of external authorizers to update their code
> accordingly.
> It looks like no deprecation is necessary, mainly because we
> already—erroneously!—treat these fields as optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5170) Adapt json creation for authorization based endpoint filtering.
[ https://issues.apache.org/jira/browse/MESOS-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307046#comment-15307046 ] Joerg Schad commented on MESOS-5170: Added authorization based filtering to /state-summary. https://reviews.apache.org/r/47559/ Used TaskObjectApprover to filter /state endpoint. https://reviews.apache.org/r/47704/ Used TaskObjectApprover to filter /tasks endpoint. https://reviews.apache.org/r/47736/ Added tests for VIEW_* authorization actions. https://reviews.apache.org/r/47977/ > Adapt json creation for authorization based endpoint filtering. > --- > > Key: MESOS-5170 > URL: https://issues.apache.org/jira/browse/MESOS-5170 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > For authorization based endpoint filtering we need to adapt the json endpoint > creation as discussed in MESOS-4931. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5403) Introduce ObjectApprover Interface to Authorizer.
[ https://issues.apache.org/jira/browse/MESOS-5403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5403: --- Summary: Introduce ObjectApprover Interface to Authorizer. (was: Introduce ObjectFilter Interface to Authorizer.) > Introduce ObjectApprover Interface to Authorizer. > - > > Key: MESOS-5403 > URL: https://issues.apache.org/jira/browse/MESOS-5403 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad >Assignee: Joerg Schad > > As outlined here > (https://docs.google.com/document/d/1FuS79P8uj5PIBycrBlkJSBKOtmeO8ezAuiNXxwIA3qA) > we plan to add the option of retrieving a FilterObject from the Authorizer > with the goal of allowing for efficient authorization of a large number of > (potentially large) objects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5404) Allow `Task` to be authorized.
Joerg Schad created MESOS-5404: -- Summary: Allow `Task` to be authorized. Key: MESOS-5404 URL: https://issues.apache.org/jira/browse/MESOS-5404 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad Assignee: Joerg Schad As we need to be able to authorize `Tasks` (e.g., for deciding whether to include them in the /state endpoint when applying authorization based filtering) we need to expose it to the authorizer. Secondly we also need to include some additional information (`user` and `Env variables`) in order to provide the authorizer with meaning information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5403) Introduce ObjectFilter Interface to Authorizer.
Joerg Schad created MESOS-5403: -- Summary: Introduce ObjectFilter Interface to Authorizer. Key: MESOS-5403 URL: https://issues.apache.org/jira/browse/MESOS-5403 Project: Mesos Issue Type: Bug Reporter: Joerg Schad Assignee: Joerg Schad As outlined here (https://docs.google.com/document/d/1FuS79P8uj5PIBycrBlkJSBKOtmeO8ezAuiNXxwIA3qA) we plan to add the option of retrieving a FilterObject from the Authorizer with the goal of allowing for efficient authorization of a large number of (potentially large) objects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5379) Authentication documentation for libprocess endpoints can be misleading.
[
https://issues.apache.org/jira/browse/MESOS-5379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282558#comment-15282558
]
Joerg Schad commented on MESOS-5379:
We discussed when introducing that Macro that we might want to extend the
message (e.g., making it a custom string, enum) for allowing a more detailed
Message later on.
> Authentication documentation for libprocess endpoints can be misleading.
>
>
> Key: MESOS-5379
> URL: https://issues.apache.org/jira/browse/MESOS-5379
> Project: Mesos
> Issue Type: Bug
> Components: documentation, libprocess
>Affects Versions: 0.29.0
>Reporter: Benjamin Bannier
>Priority: Blocker
> Labels: mesosphere, tech-debt
> Fix For: 0.29.0
>
>
> Libprocess exposes a number of endpoints (at least: {{/logging}},
> {{/metrics}}, and {{/profiler}}). If libprocess was initialized with some
> realm these endpoints require authentication, and don't if not.
> To generate endpoint help we currently use the also function
> {{AUTHENTICATION}} which injects the following into the help string,
> {code}
> This endpoints requires authentication iff HTTP authentication is enabled.
> {code}
> with {{iff}} documenting a coupling stronger between required authentication
> and enabled authentication which might not be true for above libprocess
> endpoints -- it is e.g., true when these endpoints are exposed through mesos
> masters/agents, but possibly not if exposed through other executables.
> It seems for libprocess endpoint a less strong formulation like e.g.,
> {code}
> This endpoints supports authentication. If HTTP authentication is enabled,
> this endpoint may require authentication.
> {code}
> might make the generated help strings more reusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276477#comment-15276477 ] Joerg Schad commented on MESOS-5168: and the current prototype used to benchmark here: https://github.com/joerg84/mesos/tree/filterPrototype > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276475#comment-15276475 ] Joerg Schad commented on MESOS-5168: Benchmark results can be found here: https://docs.google.com/document/d/1Ojq55I_2iyYWMSxnq9TvshVn9JGwEX4PfRBkwgZTiuY > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5292) Website references non-existing file NewbieContributionOverview.jpg
[
https://issues.apache.org/jira/browse/MESOS-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-5292:
---
Attachment: Screen Shot 2016-05-09 at 12.07.09.png
Broken rendered page.
> Website references non-existing file NewbieContributionOverview.jpg
> ---
>
> Key: MESOS-5292
> URL: https://issues.apache.org/jira/browse/MESOS-5292
> Project: Mesos
> Issue Type: Bug
> Components: project website
>Reporter: Benjamin Bannier
> Labels: site
> Attachments: Screen Shot 2016-05-09 at 12.07.09.png
>
>
> The website references the non-existing file
> {{NewbieContributionOverview.jpg}} in {{docs/newbie-guide.md}}. Looking at
> the commit adding this documentation it appears this file was never added to
> the repository. We should either provide the file or rewrite the section to
> work without needing to reference it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5292) Website references non-existing file NewbieContributionOverview.jpg
[
https://issues.apache.org/jira/browse/MESOS-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276172#comment-15276172
]
Joerg Schad commented on MESOS-5292:
I created https://reviews.apache.org/r/47115/ to remove the reference to the
image for now (feel free to re-add after the image has been added).
The review also fixes some other style issue with the more strict website
markdown renderer (e.g., broken links etc).
When doing changes to the documentation please check the documentation with the
website docker generator
(https://github.com/apache/mesos/blob/master/support/site-docker/README.md).
> Website references non-existing file NewbieContributionOverview.jpg
> ---
>
> Key: MESOS-5292
> URL: https://issues.apache.org/jira/browse/MESOS-5292
> Project: Mesos
> Issue Type: Bug
> Components: project website
>Reporter: Benjamin Bannier
> Labels: site
>
> The website references the non-existing file
> {{NewbieContributionOverview.jpg}} in {{docs/newbie-guide.md}}. Looking at
> the commit adding this documentation it appears this file was never added to
> the repository. We should either provide the file or rewrite the section to
> work without needing to reference it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5337) Add Master Flag to enable fine-grained filtering of HTTP endpoints.
[ https://issues.apache.org/jira/browse/MESOS-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275145#comment-15275145 ] Joerg Schad commented on MESOS-5337: Then we still incur the performance penalty. But if acls are permissive it would mean that the user cannot see any frameworks/tasks... I will add a warning in that case. > Add Master Flag to enable fine-grained filtering of HTTP endpoints. > --- > > Key: MESOS-5337 > URL: https://issues.apache.org/jira/browse/MESOS-5337 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > > As the fine-grained filtering of endpoints can the rather expensive, we > should create a master flag to enable/disable this feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5339) Create Tests for testing fine-grained HTTP endpoint filtering.
Joerg Schad created MESOS-5339: -- Summary: Create Tests for testing fine-grained HTTP endpoint filtering. Key: MESOS-5339 URL: https://issues.apache.org/jira/browse/MESOS-5339 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad Assignee: Joerg Schad -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5337) Add Master Flag to enable fine-grained filtering of HTTP endpoints.
[ https://issues.apache.org/jira/browse/MESOS-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5337: --- Summary: Add Master Flag to enable fine-grained filtering of HTTP endpoints. (was: Add Master Flag to enable ) > Add Master Flag to enable fine-grained filtering of HTTP endpoints. > --- > > Key: MESOS-5337 > URL: https://issues.apache.org/jira/browse/MESOS-5337 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > > As the fine-grained filtering of endpoints can the rather expensive, we > should create a master flag to enable/disable this feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5338) Add `user` to `Task` protobuf message.
Joerg Schad created MESOS-5338: -- Summary: Add `user` to `Task` protobuf message. Key: MESOS-5338 URL: https://issues.apache.org/jira/browse/MESOS-5338 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad Assignee: Joerg Schad The LocalAuthorizer is supposed to use the OS `user` under which tasks are running for authorization. As the master keeps track of running and completed processes we need access to this information in Task in order to authorize such tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5170) Adapt json creation for authorization based endpoint filtering.
[ https://issues.apache.org/jira/browse/MESOS-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5170: --- Shepherd: Michael Park (was: Adam B) > Adapt json creation for authorization based endpoint filtering. > --- > > Key: MESOS-5170 > URL: https://issues.apache.org/jira/browse/MESOS-5170 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > For authorization based endpoint filtering we need to adapt the json endpoint > creation as discussed in MESOS-4931. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5337) Add Master Flag to enable
Joerg Schad created MESOS-5337: -- Summary: Add Master Flag to enable Key: MESOS-5337 URL: https://issues.apache.org/jira/browse/MESOS-5337 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad Assignee: Joerg Schad As the fine-grained filtering of endpoints can the rather expensive, we should create a master flag to enable/disable this feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5169) Introduce new Authorizer Actions for Authorized based filtering of endpoints.
[ https://issues.apache.org/jira/browse/MESOS-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274686#comment-15274686 ] Joerg Schad commented on MESOS-5169: That one is discarded as we now simply use the value field and protobuf `SerializeToString`. > Introduce new Authorizer Actions for Authorized based filtering of endpoints. > - > > Key: MESOS-5169 > URL: https://issues.apache.org/jira/browse/MESOS-5169 > Project: Mesos > Issue Type: Improvement > Components: security >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > For authorization based endpoint filtering we need to introduce the > authorizer actions outlined via MESOS-4932. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5273) Need support for Authorization information via HELP.
[ https://issues.apache.org/jira/browse/MESOS-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5273: --- Description: We should add information about authentication to the help message and thereby endpoint documentation (similarly as MESOS-4934 has done for authentication). (was: MESOS-4785 reorganizes the documentation of the authorization features that are available in Mesos. The authorization of the "/flags" endpoint, introduced in MESOS-5142 needs to be documented in a similar way.) > Need support for Authorization information via HELP. > > > Key: MESOS-5273 > URL: https://issues.apache.org/jira/browse/MESOS-5273 > Project: Mesos > Issue Type: Improvement > Components: documentation >Reporter: Jan Schlicht >Assignee: Jan Schlicht > Labels: documentation, mesosphere, security > Fix For: 0.29.0 > > > We should add information about authentication to the help message and > thereby endpoint documentation (similarly as MESOS-4934 has done for > authentication). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5273) Need support for Authorization information via HELP.
[ https://issues.apache.org/jira/browse/MESOS-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad reassigned MESOS-5273: -- Assignee: Joerg Schad (was: Jan Schlicht) > Need support for Authorization information via HELP. > > > Key: MESOS-5273 > URL: https://issues.apache.org/jira/browse/MESOS-5273 > Project: Mesos > Issue Type: Improvement > Components: documentation >Reporter: Jan Schlicht >Assignee: Joerg Schad > Labels: documentation, mesosphere, security > Fix For: 0.29.0 > > > We should add information about authentication to the help message and > thereby endpoint documentation (similarly as MESOS-4934 has done for > authentication). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5273) Need support for Authorization information via HELP.
[ https://issues.apache.org/jira/browse/MESOS-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5273: --- Summary: Need support for Authorization information via HELP. (was: Document "/flags" endpoint authorization as in MESOS-4785) > Need support for Authorization information via HELP. > > > Key: MESOS-5273 > URL: https://issues.apache.org/jira/browse/MESOS-5273 > Project: Mesos > Issue Type: Improvement > Components: documentation >Reporter: Jan Schlicht >Assignee: Jan Schlicht > Labels: documentation, mesosphere, security > Fix For: 0.29.0 > > > MESOS-4785 reorganizes the documentation of the authorization features that > are available in Mesos. The authorization of the "/flags" endpoint, > introduced in MESOS-5142 needs to be documented in a similar way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5313) Failed to set quota and update weight according to document
[
https://issues.apache.org/jira/browse/MESOS-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266886#comment-15266886
]
Joerg Schad commented on MESOS-5313:
+1, IMO we can change that to file but should make that consistent (e.g.,
changing jsonMessageBody to quota.json or similar). See my comments on the
review.
> Failed to set quota and update weight according to document
> ---
>
> Key: MESOS-5313
> URL: https://issues.apache.org/jira/browse/MESOS-5313
> Project: Mesos
> Issue Type: Documentation
> Components: documentation
>Affects Versions: 0.27.2, 0.28.1
>Reporter: Guangya Liu
>Assignee: Guangya Liu
>Priority: Minor
> Labels: documentation, mesosphere
>
> {code}
> root@mesos002:~/test# curl -d jsonMessageBody -X POST
> http://192.168.56.12:5050/quota
> Failed to parse set quota request JSON 'jsonMessageBody': syntax error at
> line 1 near: jsonMessageBodyroot@mesos002:~/test# cat jsonMessageBody
> {
> "role": "role1",
> "guarantee": [{
> "name": "cpus",
> "type": "SCALAR",
> "scalar": {
> "value": 1
> }
> }, {
> "name": "mem",
> "type": "SCALAR",
> "scalar": {
> "value": 128
> }
> }]
> }
> root@mesos002:~/test# curl -d weight.json -X PUT
> http://192.168.56.12:5050/weights
> Failed to parse update weights request JSON ('weight.json'): syntax error at
> line 1 near: weight.js
> root@mesos002:~/test# cat weight.json
> [
> {
> "role": "role1",
> "weight": 2.0
> },
> {
> "role": "role2",
> "weight": 3.5
> }
> ]
> {code}
> The right command should be adding {{@}} before the quota json file
> {{jsonMessageBody}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-5304) /metrics/snapshot endpoint help disappeared on agent.
Joerg Schad created MESOS-5304: -- Summary: /metrics/snapshot endpoint help disappeared on agent. Key: MESOS-5304 URL: https://issues.apache.org/jira/browse/MESOS-5304 Project: Mesos Issue Type: Bug Reporter: Joerg Schad Assignee: Greg Mann After https://github.com/apache/mesos/commit/066fc4bd0df6690a5e1a929d3836e307c1e22586 the help for the /metrics/snapshot endpoint on the agent doesn't appear anymore (Master endpoint help is unchanged). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5170) Adapt json creation for authorization based endpoint filtering.
[ https://issues.apache.org/jira/browse/MESOS-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5170: --- Story Points: 5 (was: 3) Labels: authorization mesosphere security (was: mesosphere security) > Adapt json creation for authorization based endpoint filtering. > --- > > Key: MESOS-5170 > URL: https://issues.apache.org/jira/browse/MESOS-5170 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad > Labels: authorization, mesosphere, security > > For authorization based endpoint filtering we need to adapt the json endpoint > creation as discussed in MESOS-4931. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5169) Introduce new Authorizer Actions for Authorized based filtering of endpoints.
[ https://issues.apache.org/jira/browse/MESOS-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5169: --- Labels: authorization mesosphere security (was: ) > Introduce new Authorizer Actions for Authorized based filtering of endpoints. > - > > Key: MESOS-5169 > URL: https://issues.apache.org/jira/browse/MESOS-5169 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad > Labels: authorization, mesosphere, security > > For authorization based endpoint filtering we need to introduce the > authorizer actions outlined via MESOS-493. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5168: --- Labels: authorization mesosphere security (was: mesosphere security) > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad > Labels: authorization, mesosphere, security > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5168) Benchmark overhead of authorization based filtering.
Joerg Schad created MESOS-5168: -- Summary: Benchmark overhead of authorization based filtering. Key: MESOS-5168 URL: https://issues.apache.org/jira/browse/MESOS-5168 Project: Mesos Issue Type: Improvement Reporter: Joerg Schad When adding authorization based filtering as outlined in MESOS-4931 we need to be careful especially for performance critical endpoints such as /state. We should ensure via a benchmark that performance does not degreade below an acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5168: --- Labels: mesosphere security (was: ) > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad > Labels: mesosphere, security > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5170) Adapt json creation for authorization based endpoint filtering.
[ https://issues.apache.org/jira/browse/MESOS-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-5170: --- Labels: mesosphere security (was: mesosphere) > Adapt json creation for authorization based endpoint filtering. > --- > > Key: MESOS-5170 > URL: https://issues.apache.org/jira/browse/MESOS-5170 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad > Labels: mesosphere, security > > For authorization based endpoint filtering we need to adapt the json endpoint > creation as discussed in MESOS-4931. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
