[jira] [Updated] (MESOS-5295) The task launched by non-checkpointed HTTP command executor will keep running till executor shutdown grace period (5s) after agent process exits.
[
https://issues.apache.org/jira/browse/MESOS-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-5295:
--
Summary: The task launched by non-checkpointed HTTP command executor will
keep running till executor shutdown grace period (5s) after agent process
exits. (was: The task launched by non-checkpointed HTTP command executor will
keep running after agent is done)
> The task launched by non-checkpointed HTTP command executor will keep running
> till executor shutdown grace period (5s) after agent process exits.
> -
>
> Key: MESOS-5295
> URL: https://issues.apache.org/jira/browse/MESOS-5295
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Reporter: Qian Zhang
>Assignee: Qian Zhang
>
> When I test HTTP command executor, I found an issue, here is my steps:
> 1. A framework which has no checkpoint enabled launches a long running task
> (e.g., sleep 1000).
> 2. After the task is running, kill the agent.
>
> Then I see the HTTP command executor will terminate after 5s
> ("DEFAULT_EXECUTOR_SHUTDOWN_GRACE_PERIOD"), but the task will always run.
> This behavior is not consistent with driver based command executor: after
> agent is killed, that executor will kill the task and then self terminate
> after 1s (there is a "os::sleep(Seconds(1));" in "reaped()").
> The root cause of this difference is, for driver based command executor, when
> the driver found agent is down, it will call executor->shutdown()
> (https://github.com/apache/mesos/blob/0.28.1/src/exec/exec.cpp#L487), so the
> executor will kill the task and then self terminate. But for HTTP command
> executor, its "disconnected()" will be called
> (https://github.com/apache/mesos/blob/0.28.1/src/executor/executor.cpp#L388)
> when agent is down, and currently we do not do anything in its
> "disconnected()", so the task will keep running and the executor will be
> killed after 5s
> (https://github.com/apache/mesos/blob/0.28.1/src/executor/executor.cpp#L623).
> The behavior of driver based command executor is correct, we need to make
> sure HTTP command executor kills the task when agent is down if checkpoint is
> not enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4973) Duplicates in 'unregistered_frameworks' in /state
[ https://issues.apache.org/jira/browse/MESOS-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261446#comment-15261446 ] shijinkui commented on MESOS-4973: -- for the `unregistered_frameworks `, how do we treat them, clear all the 'unregistered_frameworks'? > Duplicates in 'unregistered_frameworks' in /state > -- > > Key: MESOS-4973 > URL: https://issues.apache.org/jira/browse/MESOS-4973 > Project: Mesos > Issue Type: Bug >Reporter: Yan Xu >Priority: Minor > > In our clusters where many frameworks run, 'unregistered_frameworks' > currently doesn't show what it semantically means, but rather "a list of > frameworkIDs for each orphaned task", which means a lot of duplicated > frameworkIDs. > For this filed to be useful we need to deduplicate when outputting the list. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1104) Move linux/fs.hpp out of `mesos` namespace in linux/fs.h
[
https://issues.apache.org/jira/browse/MESOS-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261404#comment-15261404
]
Michael Park commented on MESOS-1104:
-
[~xds2000] It looks like this patch is simply moving everything in
{{mesos::internal::fs}} to {{fs}}. That is, it doesn't look like we're
consolidating multiple implementations. Again, what are we trying to accomplish
here? Is there some place where we want access to these functions but for some
reason, are not accessible or something?
> Move linux/fs.hpp out of `mesos` namespace in linux/fs.h
>
>
> Key: MESOS-1104
> URL: https://issues.apache.org/jira/browse/MESOS-1104
> Project: Mesos
> Issue Type: Improvement
>Reporter: Archana kumari
>Assignee: Deshi Xiao
> Labels: mesosphere, newbie
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-5296) Split Resource and Inverse offer protobufs for V1 API
Joris Van Remoortere created MESOS-5296: --- Summary: Split Resource and Inverse offer protobufs for V1 API Key: MESOS-5296 URL: https://issues.apache.org/jira/browse/MESOS-5296 Project: Mesos Issue Type: Improvement Reporter: Joris Van Remoortere Assignee: Joris Van Remoortere Fix For: 0.29.0 The protobufs for the V1 api regarding inverse offers initially re-used the existing offer / rescind / accept / decline messages for regular offers. We should split these out the be more explicit, and provide the ability to augment the messages with particulars to either resource or inverse offers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5295) The task launched by non-checkpointed HTTP command executor will keep running after agent is done
Qian Zhang created MESOS-5295:
-
Summary: The task launched by non-checkpointed HTTP command
executor will keep running after agent is done
Key: MESOS-5295
URL: https://issues.apache.org/jira/browse/MESOS-5295
Project: Mesos
Issue Type: Bug
Components: HTTP API
Reporter: Qian Zhang
Assignee: Qian Zhang
When I test HTTP command executor, I found an issue, here is my steps:
1. A framework which has no checkpoint enabled launches a long running task
(e.g., sleep 1000).
2. After the task is running, kill the agent.
Then I see the HTTP command executor will terminate after 5s
("DEFAULT_EXECUTOR_SHUTDOWN_GRACE_PERIOD"), but the task will always run. This
behavior is not consistent with driver based command executor: after agent is
killed, that executor will kill the task and then self terminate after 1s
(there is a "os::sleep(Seconds(1));" in "reaped()").
The root cause of this difference is, for driver based command executor, when
the driver found agent is down, it will call executor->shutdown()
(https://github.com/apache/mesos/blob/0.28.1/src/exec/exec.cpp#L487), so the
executor will kill the task and then self terminate. But for HTTP command
executor, its "disconnected()" will be called
(https://github.com/apache/mesos/blob/0.28.1/src/executor/executor.cpp#L388)
when agent is down, and currently we do not do anything in its
"disconnected()", so the task will keep running and the executor will be killed
after 5s
(https://github.com/apache/mesos/blob/0.28.1/src/executor/executor.cpp#L623).
The behavior of driver based command executor is correct, we need to make sure
HTTP command executor kills the task when agent is down if checkpoint is not
enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5294) Status updates after a health check are incomplete or invalid
[
https://issues.apache.org/jira/browse/MESOS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261107#comment-15261107
]
Joseph Wu commented on MESOS-5294:
--
AFAIK, the function of Mesos-DNS is not dependent on the health of the task.
It simply looks at the masters {{/state}} endpoint and creates DNS records for
each master/agent/framework/task.
Can you include your Marathon app definition? (And, perhaps, the result of
{{master/state}}, filtered down to the task in question.)
> Status updates after a health check are incomplete or invalid
> -
>
> Key: MESOS-5294
> URL: https://issues.apache.org/jira/browse/MESOS-5294
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns,
> ubuntu 14.04
>Reporter: Travis Hegner
>Assignee: Travis Hegner
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> With command health checks enabled via marathon, mesos-dns will resolve the
> task correctly until the task is reported as "healthy". At that point,
> mesos-dns stops resolving the task correctly.
> Digging through src/docker/executor.cpp, I found that in the
> "taskHealthUpdated()" function is attempting to copy the taskID to the new
> status instance with "status.mutable_task_id()->CopyFrom(taskID);", but other
> instances of status updates have a similar line
> "status.mutable_task_id()->CopyFrom(taskID.get());".
> My assumption is that this difference is causing the status update after a
> health check to not have a proper taskID, which in turn is causing an
> incorrect state.json output.
> I'll try to get a patch together soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5294) Status updates after a health check are incomplete or invalid
[ https://issues.apache.org/jira/browse/MESOS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260924#comment-15260924 ] Gilbert Song commented on MESOS-5294: - [~thegner], thanks for reporting this issue. I am thinking is this issue occurred specific for docker 1.11 or mesos 0.28.0(it doesn't seem any change on mesos docker executor on 0.28 release though). > Status updates after a health check are incomplete or invalid > - > > Key: MESOS-5294 > URL: https://issues.apache.org/jira/browse/MESOS-5294 > Project: Mesos > Issue Type: Bug > Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns, > ubuntu 14.04 >Reporter: Travis Hegner >Assignee: Travis Hegner > Original Estimate: 2h > Remaining Estimate: 2h > > With command health checks enabled via marathon, mesos-dns will resolve the > task correctly until the task is reported as "healthy". At that point, > mesos-dns stops resolving the task correctly. > Digging through src/docker/executor.cpp, I found that in the > "taskHealthUpdated()" function is attempting to copy the taskID to the new > status instance with "status.mutable_task_id()->CopyFrom(taskID);", but other > instances of status updates have a similar line > "status.mutable_task_id()->CopyFrom(taskID.get());". > My assumption is that this difference is causing the status update after a > health check to not have a proper taskID, which in turn is causing an > incorrect state.json output. > I'll try to get a patch together soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3765) Make offer size adjustable (granularity)
[
https://issues.apache.org/jira/browse/MESOS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rukletsov updated MESOS-3765:
---
Assignee: (was: Guangya Liu)
> Make offer size adjustable (granularity)
>
>
> Key: MESOS-3765
> URL: https://issues.apache.org/jira/browse/MESOS-3765
> Project: Mesos
> Issue Type: Improvement
> Components: allocation
>Reporter: Alexander Rukletsov
>Priority: Minor
>
> The built-in allocator performs "coarse-grained" allocation, meaning that it
> always allocates the entire remaining agent resources to a single framework.
> This may heavily impact allocation fairness in some cases, for example in
> presence of numerous greedy frameworks and a small number of powerful agents.
> A possible solution would be to allow operators explicitly specify
> granularity via allocator flags. While this can be tricky for non-standard
> resources, it's pretty straightforward for {{cpus}} and {{mem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3765) Make offer size adjustable (granularity)
[
https://issues.apache.org/jira/browse/MESOS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rukletsov updated MESOS-3765:
---
Priority: Minor (was: Major)
> Make offer size adjustable (granularity)
>
>
> Key: MESOS-3765
> URL: https://issues.apache.org/jira/browse/MESOS-3765
> Project: Mesos
> Issue Type: Improvement
> Components: allocation
>Reporter: Alexander Rukletsov
>Assignee: Guangya Liu
>Priority: Minor
>
> The built-in allocator performs "coarse-grained" allocation, meaning that it
> always allocates the entire remaining agent resources to a single framework.
> This may heavily impact allocation fairness in some cases, for example in
> presence of numerous greedy frameworks and a small number of powerful agents.
> A possible solution would be to allow operators explicitly specify
> granularity via allocator flags. While this can be tricky for non-standard
> resources, it's pretty straightforward for {{cpus}} and {{mem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5294) Status updates after a health check are incomplete or invalid
[ https://issues.apache.org/jira/browse/MESOS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Hegner updated MESOS-5294: - Summary: Status updates after a health check are incomplete or invalid (was: Status updates after a health check and incomplete) > Status updates after a health check are incomplete or invalid > - > > Key: MESOS-5294 > URL: https://issues.apache.org/jira/browse/MESOS-5294 > Project: Mesos > Issue Type: Bug > Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns, > ubuntu 14.04 >Reporter: Travis Hegner >Assignee: Travis Hegner > Original Estimate: 2h > Remaining Estimate: 2h > > With command health checks enabled via marathon, mesos-dns will resolve the > task correctly until the task is reported as "healthy". At that point, > mesos-dns stops resolving the task correctly. > Digging through src/docker/executor.cpp, I found that in the > "taskHealthUpdated()" function is attempting to copy the taskID to the new > status instance with "status.mutable_task_id()->CopyFrom(taskID);", but other > instances of status updates have a similar line > "status.mutable_task_id()->CopyFrom(taskID.get());". > My assumption is that this difference is causing the status update after a > health check to not have a proper taskID, which in turn is causing an > incorrect state.json output. > I'll try to get a patch together soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5294) Status updates after a health check and incomplete
Travis Hegner created MESOS-5294: Summary: Status updates after a health check and incomplete Key: MESOS-5294 URL: https://issues.apache.org/jira/browse/MESOS-5294 Project: Mesos Issue Type: Bug Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns, ubuntu 14.04 Reporter: Travis Hegner Assignee: Travis Hegner With command health checks enabled via marathon, mesos-dns will resolve the task correctly until the task is reported as "healthy". At that point, mesos-dns stops resolving the task correctly. Digging through src/docker/executor.cpp, I found that in the "taskHealthUpdated()" function is attempting to copy the taskID to the new status instance with "status.mutable_task_id()->CopyFrom(taskID);", but other instances of status updates have a similar line "status.mutable_task_id()->CopyFrom(taskID.get());". My assumption is that this difference is causing the status update after a health check to not have a proper taskID, which in turn is causing an incorrect state.json output. I'll try to get a patch together soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5174) Update the balloon-framework to run on test clusters
[
https://issues.apache.org/jira/browse/MESOS-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-5174:
--
Sprint: Mesosphere Sprint 33 (was: Mesosphere Sprint 33, Mesosphere Sprint
34)
> Update the balloon-framework to run on test clusters
>
>
> Key: MESOS-5174
> URL: https://issues.apache.org/jira/browse/MESOS-5174
> Project: Mesos
> Issue Type: Improvement
> Components: framework, technical debt
>Reporter: Joseph Wu
>Assignee: Joseph Wu
> Labels: mesosphere, tech-debt
>
> There are a couple of problems with the balloon framework that prevent it
> from being deployed (easily) on an actual cluster:
> * The framework accepts 100% of memory in an offer. This means the expected
> behavior (finish or OOM) is dependent on the offer size.
> * The framework assumes the {{balloon-executor}} binary is available on each
> agent. This is generally only true in the build environment or in
> single-agent test environments.
> * The framework does not specify CPUs with the executor. This is required by
> many isolators.
> * The executor's {{TASK_FINISHED}} logic path was untested and is flaky.
> * The framework has no metrics.
> * The framework only launches a single task and then exits. With this
> behavior, we can't have useful metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
[
https://issues.apache.org/jira/browse/MESOS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260606#comment-15260606
]
Anand Mazumdar commented on MESOS-5293:
---
This has been there for a long time before we implemented the executor
endpoint. It's just that the example mentions them. We would need to fix this
for all the slave endpoints if it turns out to be a bug. (haven't read the
description fully yet)
> Endpoint handlers for master and agent need to be implemented surprisingly
> differently
> --
>
> Key: MESOS-5293
> URL: https://issues.apache.org/jira/browse/MESOS-5293
> Project: Mesos
> Issue Type: Bug
> Components: master, slave
>Reporter: Benjamin Bannier
> Labels: tech-debt
>
> The way endpoints are routed is inconsistent between master and agent code
> which makes adding or extending handlers error prone.
> In the master we route like this:
> {code}
> // Setup HTTP routes.
> route("/api/v1/scheduler",
> DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
> Http::SCHEDULER_HELP(),
> [this](const process::http::Request& request,
> const Option& principal) {
> Http::log(request);
> return http.scheduler(request, principal);
> });
> {code}
> We capture a pointer to the current {{Master}} in the callback which allows
> us to access master state from an endpoint handler. The handler is a
> (probably non-static) member function of the master's member variable
> {{http}} which outlives the callback.
> Routing in the agent looks like this:
> {code}
> // Setup HTTP routes.
> Http http = Http(this);
> route("/api/v1/executor",
> Http::EXECUTOR_HELP(),
> [http](const process::http::Request& request) {
> Http::log(request);
> return http.executor(request);
> });
> {code}
> In contrast to the master code we here copy a {{Http}} by value into the
> callback. Since the callback is currently treated like a value and might
> e.g., be freely copied around we are only guaranteed that {{http}} lives long
> enough for the handler (here {{Http::executor}}) to return. In particular,
> since endpoint handlers return a {{Future}} there is no guarantee
> that the used {{http}} lives long enough to be still valid once a
> conventional (e.g., non-static member) continuation is executed.
> Both models have their merit:
> * capturing a pointer to the master simplifies reasoning about lifetimes,
> * capturing just a {{Http}} with very short lifetime minimizes interactions
> among (e.g., concurrent) invocations of endpoint handlers.
> This great inconsistency comes with a cost though, as employing patterns
> borrowed from master endpoint handlers in agent code will lead to potentially
> subtle bugs where a developer assuming that {{http}} would outlive a
> handler's execution might introduce code invoking member functions of already
> destructed variables. This is especially likely in code employing multiple
> layers of {{delay}} or {{defer}} for which compilers seem unable to detect
> lifetime problems and emit diagnostics.
> It would be great if we could to use just one of the patterns to minimize
> confusion, e.g., the more straight-forward master pattern.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
[
https://issues.apache.org/jira/browse/MESOS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260583#comment-15260583
]
Vinod Kone commented on MESOS-5293:
---
[~anandmazumdar] I missed this in the reviews, but why do we have this
inconsistency?
> Endpoint handlers for master and agent need to be implemented surprisingly
> differently
> --
>
> Key: MESOS-5293
> URL: https://issues.apache.org/jira/browse/MESOS-5293
> Project: Mesos
> Issue Type: Bug
> Components: master, slave
>Reporter: Benjamin Bannier
> Labels: tech-debt
>
> The way endpoints are routed is inconsistent between master and agent code
> which makes adding or extending handlers error prone.
> In the master we route like this:
> {code}
> // Setup HTTP routes.
> route("/api/v1/scheduler",
> DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
> Http::SCHEDULER_HELP(),
> [this](const process::http::Request& request,
> const Option& principal) {
> Http::log(request);
> return http.scheduler(request, principal);
> });
> {code}
> We capture a pointer to the current {{Master}} in the callback which allows
> us to access master state from an endpoint handler. The handler is a
> (probably non-static) member function of the master's member variable
> {{http}} which outlives the callback.
> Routing in the agent looks like this:
> {code}
> // Setup HTTP routes.
> Http http = Http(this);
> route("/api/v1/executor",
> Http::EXECUTOR_HELP(),
> [http](const process::http::Request& request) {
> Http::log(request);
> return http.executor(request);
> });
> {code}
> In contrast to the master code we here copy a {{Http}} by value into the
> callback. Since the callback is currently treated like a value and might
> e.g., be freely copied around we are only guaranteed that {{http}} lives long
> enough for the handler (here {{Http::executor}}) to return. In particular,
> since endpoint handlers return a {{Future}} there is no guarantee
> that the used {{http}} lives long enough to be still valid once a
> conventional (e.g., non-static member) continuation is executed.
> Both models have their merit:
> * capturing a pointer to the master simplifies reasoning about lifetimes,
> * capturing just a {{Http}} with very short lifetime minimizes interactions
> among (e.g., concurrent) invocations of endpoint handlers.
> This great inconsistency comes with a cost though, as employing patterns
> borrowed from master endpoint handlers in agent code will lead to potentially
> subtle bugs where a developer assuming that {{http}} would outlive a
> handler's execution might introduce code invoking member functions of already
> destructed variables. This is especially likely in code employing multiple
> layers of {{delay}} or {{defer}} for which compilers seem unable to detect
> lifetime problems and emit diagnostics.
> It would be great if we could to use just one of the patterns to minimize
> confusion, e.g., the more straight-forward master pattern.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5286) Add authorization to libprocess HTTP endpoints
[ https://issues.apache.org/jira/browse/MESOS-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-5286: - Shepherd: Kapil Arya > Add authorization to libprocess HTTP endpoints > -- > > Key: MESOS-5286 > URL: https://issues.apache.org/jira/browse/MESOS-5286 > Project: Mesos > Issue Type: Improvement > Components: libprocess >Reporter: Greg Mann >Assignee: Greg Mann > Labels: mesosphere > Fix For: 0.29.0 > > > Now that the libprocess-level HTTP endpoints have had authentication added to > them in MESOS-4902, we can add authorization to them as well. As a first > step, we can implement a "coarse-grained" approach, in which a principal is > granted or denied access to a given endpoint. We will likely need to register > an authorizer with libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-5286) Add authorization to libprocess HTTP endpoints
[ https://issues.apache.org/jira/browse/MESOS-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann reassigned MESOS-5286: Assignee: Greg Mann > Add authorization to libprocess HTTP endpoints > -- > > Key: MESOS-5286 > URL: https://issues.apache.org/jira/browse/MESOS-5286 > Project: Mesos > Issue Type: Improvement > Components: libprocess >Reporter: Greg Mann >Assignee: Greg Mann > Labels: mesosphere > Fix For: 0.29.0 > > > Now that the libprocess-level HTTP endpoints have had authentication added to > them in MESOS-4902, we can add authorization to them as well. As a first > step, we can implement a "coarse-grained" approach, in which a principal is > granted or denied access to a given endpoint. We will likely need to register > an authorizer with libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4941) Support update existing quota.
[ https://issues.apache.org/jira/browse/MESOS-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rukletsov updated MESOS-4941: --- Sprint: Mesosphere Sprint 33 (was: Mesosphere Sprint 33, Mesosphere Sprint 34) > Support update existing quota. > -- > > Key: MESOS-4941 > URL: https://issues.apache.org/jira/browse/MESOS-4941 > Project: Mesos > Issue Type: Improvement > Components: allocation >Reporter: Zhitao Li >Assignee: Zhitao Li > Labels: Quota, mesosphere > > We want to support updating an existing quota without the cycle of delete and > recreate. This avoids the possible starvation risk of losing the quota > between delete and recreate, and also makes the interface friendly. > Design doc: > https://docs.google.com/document/d/1c8fJY9_N0W04FtUQ_b_kZM6S0eePU7eYVyfUP14dSys -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4882) Add support for command and arguments to mesos-execute.
[
https://issues.apache.org/jira/browse/MESOS-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rukletsov updated MESOS-4882:
---
Sprint: Mesosphere Sprint 33 (was: Mesosphere Sprint 33, Mesosphere Sprint
34)
> Add support for command and arguments to mesos-execute.
> ---
>
> Key: MESOS-4882
> URL: https://issues.apache.org/jira/browse/MESOS-4882
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.28.0, 0.27.2
>Reporter: Guangya Liu
>Assignee: Guangya Liu
> Labels: cli, mesosphere
>
> {{CommandInfo}} protobuf support two kinds of command:
> {code}
> // There are two ways to specify the command:
> // 1) If 'shell == true', the command will be launched via shell
> // (i.e., /bin/sh -c 'value'). The 'value' specified will be
> // treated as the shell command. The 'arguments' will be ignored.
> // 2) If 'shell == false', the command will be launched by passing
> // arguments to an executable. The 'value' specified will be
> // treated as the filename of the executable. The 'arguments'
> // will be treated as the arguments to the executable. This is
> // similar to how POSIX exec families launch processes (i.e.,
> // execlp(value, arguments(0), arguments(1), ...)).
> {code}
> The mesos-execute cannot handle 2) now, enabling 2) can help with testing and
> running one off tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5164) Add authorization to agent's /monitor/statistics endpoint.
[ https://issues.apache.org/jira/browse/MESOS-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rukletsov updated MESOS-5164: --- Summary: Add authorization to agent's /monitor/statistics endpoint. (was: Add authorization to agent's /monitor/statistics endpoint) > Add authorization to agent's /monitor/statistics endpoint. > -- > > Key: MESOS-5164 > URL: https://issues.apache.org/jira/browse/MESOS-5164 > Project: Mesos > Issue Type: Task > Components: security, slave >Reporter: Adam B >Assignee: Benjamin Bannier > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > Operators may want to enforce that only specific authorized users be able to > view per-executor resource usage statistics. For 0.29 MVP, we can make this > coarse-grained, and assume that only the operator or a operator-privileged > monitoring service will be accessing the endpoint. > For a future release, we can consider fine-grained authz that filters > statistics like we plan to do for /tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5142) Add agent flags for HTTP authorization.
[
https://issues.apache.org/jira/browse/MESOS-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rukletsov updated MESOS-5142:
---
Summary: Add agent flags for HTTP authorization. (was: Add agent flags for
HTTP authorization)
> Add agent flags for HTTP authorization.
> ---
>
> Key: MESOS-5142
> URL: https://issues.apache.org/jira/browse/MESOS-5142
> Project: Mesos
> Issue Type: Bug
> Components: security, slave
>Reporter: Jan Schlicht
>Assignee: Jan Schlicht
> Labels: mesosphere, security
> Fix For: 0.29.0
>
>
> Flags should be added to the agent to:
> 1. Enable authorization ({{--authorizers}})
> 2. Provide ACLs ({{--acls}})
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-4689) Design doc for v1 Operator API
[ https://issues.apache.org/jira/browse/MESOS-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4689: - Sprint: Mesosphere Sprint 29, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 29, Mesosphere Sprint 33) > Design doc for v1 Operator API > -- > > Key: MESOS-4689 > URL: https://issues.apache.org/jira/browse/MESOS-4689 > Project: Mesos > Issue Type: Documentation >Reporter: Vinod Kone >Assignee: Kevin Klues > > We need to design how the v1 operator API (all the HTTP endpoints exposed by > master/agent that are not for scheduler/executor interactions) looks and > works. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4233) Logging is too verbose for sysadmins / syslog
[
https://issues.apache.org/jira/browse/MESOS-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-4233:
-
Sprint: Mesosphere Sprint 26, Mesosphere Sprint 27, Mesosphere Sprint 28,
Mesosphere Sprint 29, Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere
Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
26, Mesosphere Sprint 27, Mesosphere Sprint 28, Mesosphere Sprint 29,
Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere
Sprint 33)
> Logging is too verbose for sysadmins / syslog
> -
>
> Key: MESOS-4233
> URL: https://issues.apache.org/jira/browse/MESOS-4233
> Project: Mesos
> Issue Type: Epic
>Reporter: Cody Maloney
>Assignee: Kapil Arya
> Labels: mesosphere
> Attachments: giant_port_range_logging
>
>
> Currently mesos logs a lot. When launching a thousand tasks in the space of
> 10 seconds it will print tens of thousands of log lines, overwhelming syslog
> (there is a max rate at which a process can send stuff over a unix socket)
> and not giving useful information to a sysadmin who cares about just the
> high-level activity and when something goes wrong.
> Note mesos also blocks writing to its log locations, so when writing a lot of
> log messages, it can fill up the write buffer in the kernel, and be suspended
> until the syslog agent catches up reading from the socket (GLOG does a
> blocking fwrite to stderr). GLOG also has a big mutex around logging so only
> one thing logs at a time.
> While for "internal debugging" it is useful to see things like "message went
> from internal compoent x to internal component y", from a sysadmin
> perspective I only care about the high level actions taken (launched task for
> framework x), sent offer to framework y, got task failed from host z. Note
> those are what I'd expect at the "INFO" level. At the "WARNING" level I'd
> expect very little to be logged / almost nothing in normal operation. Just
> things like "WARN: Repliacted log write took longer than expected". WARN
> would also get things like backtraces on crashes and abnormal exits / abort.
> When trying to launch 3k+ tasks inside a second, mesos logging currently
> overwhelms syslog with 100k+ messages, many of which are thousands of bytes.
> Sysadmins expect to be able to use syslog to monitor basic events in their
> system. This is too much.
> We can keep logging the messages to files, but the logging to stderr needs to
> be reduced significantly (stderr gets picked up and forwarded to syslog /
> central aggregation).
> What I would like is if I can set the stderr logging level to be different /
> independent from the file logging level (Syslog giving the "sysadmin"
> aggregated overview, files useful for debugging in depth what happened in a
> cluster). A lot of what mesos currently logs at info is really debugging info
> / should show up as debug log level.
> Some samples of mesos logging a lot more than a sysadmin would want / expect
> are attached, and some are below:
> - Every task gets printed multiple times for a basic launch:
> {noformat}
> Dec 15 22:58:30 ip-10-0-7-60.us-west-2.compute.internal mesos-master[1311]:
> I1215 22:58:29.382644 1315 master.cpp:3248] Launching task
> envy.5b19a713-a37f-11e5-8b3e-0251692d6109 of framework
> 5178f46d-71d6-422f-922c-5bbe82dff9cc- (marathon)
> Dec 15 22:58:30 ip-10-0-7-60.us-west-2.compute.internal mesos-master[1311]:
> I1215 22:58:29.382925 1315 master.hpp:176] Adding task
> envy.5b1958f2-a37f-11e5-8b3e-0251692d6109 with resources cpus(*):0.0001;
> mem(*):16; ports(*):[14047-14047]
> {noformat}
> - Every task status update prints many log lines, successful ones are part
> of normal operation and maybe should be logged at info / debug levels, but
> not to a sysadmin (Just show when things fail, and maybe aggregate counters
> to tell of the volume of working)
> - No log messagse should be really big / more than 1k characters (Would
> prevent the giant port list attached, make that easily discoverable / bug
> filable / fixable)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5232) Add capability information to ContainerInfo protobuf message.
[ https://issues.apache.org/jira/browse/MESOS-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5232: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add capability information to ContainerInfo protobuf message. > - > > Key: MESOS-5232 > URL: https://issues.apache.org/jira/browse/MESOS-5232 > Project: Mesos > Issue Type: Task > Components: containerization >Reporter: Jojy Varghese >Assignee: Jojy Varghese > Labels: mesosphere > > To enable support for capability as first class framework entity, we need to > add capabilities related information to the ContainerInfo protobuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4908) Tasks cannot be killed forcefully.
[ https://issues.apache.org/jira/browse/MESOS-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4908: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Tasks cannot be killed forcefully. > -- > > Key: MESOS-4908 > URL: https://issues.apache.org/jira/browse/MESOS-4908 > Project: Mesos > Issue Type: Improvement >Reporter: Alexander Rukletsov >Assignee: Alexander Rukletsov > Labels: mesosphere > > Currently there is no way for a scheduler to instruct the executor to kill a > certain task immediately, skipping any possible timeouts and / or kill > policies. This may be desirable in cases like, e.g., the kill policy is 10 > minutes but something went wrong, so the scheduler decides to issue a > forceful kill. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5164) Add authorization to agent's /monitor/statistics endpoint
[ https://issues.apache.org/jira/browse/MESOS-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5164: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add authorization to agent's /monitor/statistics endpoint > - > > Key: MESOS-5164 > URL: https://issues.apache.org/jira/browse/MESOS-5164 > Project: Mesos > Issue Type: Task > Components: security, slave >Reporter: Adam B >Assignee: Benjamin Bannier > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > Operators may want to enforce that only specific authorized users be able to > view per-executor resource usage statistics. For 0.29 MVP, we can make this > coarse-grained, and assume that only the operator or a operator-privileged > monitoring service will be accessing the endpoint. > For a future release, we can consider fine-grained authz that filters > statistics like we plan to do for /tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4941) Support update existing quota.
[ https://issues.apache.org/jira/browse/MESOS-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4941: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Support update existing quota. > -- > > Key: MESOS-4941 > URL: https://issues.apache.org/jira/browse/MESOS-4941 > Project: Mesos > Issue Type: Improvement > Components: allocation >Reporter: Zhitao Li >Assignee: Zhitao Li > Labels: Quota, mesosphere > > We want to support updating an existing quota without the cycle of delete and > recreate. This avoids the possible starvation risk of losing the quota > between delete and recreate, and also makes the interface friendly. > Design doc: > https://docs.google.com/document/d/1c8fJY9_N0W04FtUQ_b_kZM6S0eePU7eYVyfUP14dSys -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5228) Add tests for Capability API.
[ https://issues.apache.org/jira/browse/MESOS-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5228: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add tests for Capability API. > - > > Key: MESOS-5228 > URL: https://issues.apache.org/jira/browse/MESOS-5228 > Project: Mesos > Issue Type: Task > Components: containerization >Reporter: Jojy Varghese >Assignee: Jojy Varghese > Labels: mesosphere, unified-containerizer-mvp > > Add basic tests for the capability API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4938) Support docker registry authentication
[ https://issues.apache.org/jira/browse/MESOS-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4938: - Sprint: Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33) > Support docker registry authentication > -- > > Key: MESOS-4938 > URL: https://issues.apache.org/jira/browse/MESOS-4938 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Gilbert Song > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5169) Introduce new Authorizer Actions for Authorized based filtering of endpoints.
[ https://issues.apache.org/jira/browse/MESOS-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5169: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Introduce new Authorizer Actions for Authorized based filtering of endpoints. > - > > Key: MESOS-5169 > URL: https://issues.apache.org/jira/browse/MESOS-5169 > Project: Mesos > Issue Type: Improvement > Components: security >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > For authorization based endpoint filtering we need to introduce the > authorizer actions outlined via MESOS-4932. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4781) Executor env variables should not be leaked to the command task.
[ https://issues.apache.org/jira/browse/MESOS-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4781: - Sprint: Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33) > Executor env variables should not be leaked to the command task. > > > Key: MESOS-4781 > URL: https://issues.apache.org/jira/browse/MESOS-4781 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Gilbert Song >Assignee: Gilbert Song > Labels: mesosphere > > Currently, command task inherits the env variables of the command executor. > This is less ideal because the command executor environment variables include > some Mesos internal env variables like MESOS_XXX and LIBPROCESS_XXX. Also, > this behavior does not match what Docker containerizer does. We should > construct the env variables from scratch for the command task, rather than > relying on inheriting the env variables from the command executor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5101) Add CMake build to docker_build.sh
[ https://issues.apache.org/jira/browse/MESOS-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5101: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add CMake build to docker_build.sh > -- > > Key: MESOS-5101 > URL: https://issues.apache.org/jira/browse/MESOS-5101 > Project: Mesos > Issue Type: Improvement >Reporter: Juan Larriba >Assignee: Juan Larriba > > Add the CMake build system to docker_build.sh to automatically test the build > on Jenkins alongside gcc and clang. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5271) Add alias support for Flags
[ https://issues.apache.org/jira/browse/MESOS-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5271: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add alias support for Flags > --- > > Key: MESOS-5271 > URL: https://issues.apache.org/jira/browse/MESOS-5271 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Vinod Kone > > Currently there is no support for a flag to have an alias. Such support would > be useful to rename/deprecate a flag. > For example, for MESOS-4386, we could let the flag have `--authenticate` name > and a `--authenticate_frameworks` alias. The alias can be marked as > deprecated (need to add support for this as well). > This support will also be useful for slave/agent flag rename. See MESOS-3781 > for details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3739) Mesos does not set Content-Type for 400 Bad Request
[
https://issues.apache.org/jira/browse/MESOS-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-3739:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Mesos does not set Content-Type for 400 Bad Request
> ---
>
> Key: MESOS-3739
> URL: https://issues.apache.org/jira/browse/MESOS-3739
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0, 0.24.1, 0.25.0
>Reporter: Ben Whitehead
>Assignee: Vinod Kone
> Labels: mesosphere
>
> While integrating with the HTTP Scheduler API I encountered the following
> scenario.
> The message below was serialized to protobuf and sent as the POST body
> {code:title=message}
> call {
> type: ACKNOWLEDGE,
> acknowledge: {
> uuid: ,
> agentID: { value: "20151012-182734-16777343-5050-8978-S2" },
> taskID: { value: "task-1" }
> }
> }
> {code}
> {code:title=Request Headers}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/x-protobuf
> Content-Length: 73
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> I received the following response
> {code:title=Response Headers}
> HTTP/1.1 400 Bad Request
> Date: Wed, 14 Oct 2015 23:21:36 GMT
> Content-Length: 74
> Failed to validate Scheduler::Call: Expecting 'framework_id' to be present
> {code}
> Even though my accept header made no mention of {{text/plain}} the message
> body returned to me is {{text/plain}}. Additionally, there is no
> {{Content-Type}} header set on the response so I can't even do anything
> intelligently in my response handler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-4434) Install 3rdparty package boost, glog, protobuf and picojson when installing Mesos
[ https://issues.apache.org/jira/browse/MESOS-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4434: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Install 3rdparty package boost, glog, protobuf and picojson when installing > Mesos > - > > Key: MESOS-4434 > URL: https://issues.apache.org/jira/browse/MESOS-4434 > Project: Mesos > Issue Type: Bug > Components: build, modules >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > Fix For: 0.29.0 > > > Mesos modules depend on having these packages installed with the exact > version as Mesos was compiled with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4882) Add support for command and arguments to mesos-execute.
[
https://issues.apache.org/jira/browse/MESOS-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-4882:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Add support for command and arguments to mesos-execute.
> ---
>
> Key: MESOS-4882
> URL: https://issues.apache.org/jira/browse/MESOS-4882
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.28.0, 0.27.2
>Reporter: Guangya Liu
>Assignee: Guangya Liu
> Labels: cli, mesosphere
>
> {{CommandInfo}} protobuf support two kinds of command:
> {code}
> // There are two ways to specify the command:
> // 1) If 'shell == true', the command will be launched via shell
> // (i.e., /bin/sh -c 'value'). The 'value' specified will be
> // treated as the shell command. The 'arguments' will be ignored.
> // 2) If 'shell == false', the command will be launched by passing
> // arguments to an executable. The 'value' specified will be
> // treated as the filename of the executable. The 'arguments'
> // will be treated as the arguments to the executable. This is
> // similar to how POSIX exec families launch processes (i.e.,
> // execlp(value, arguments(0), arguments(1), ...)).
> {code}
> The mesos-execute cannot handle 2) now, enabling 2) can help with testing and
> running one off tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3781) Replace Master/Slave Terminology Phase I - Add duplicate agent flags
[ https://issues.apache.org/jira/browse/MESOS-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-3781: - Sprint: Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 32, Mesosphere Sprint 33) > Replace Master/Slave Terminology Phase I - Add duplicate agent flags > - > > Key: MESOS-3781 > URL: https://issues.apache.org/jira/browse/MESOS-3781 > Project: Mesos > Issue Type: Task >Reporter: Diana Arroyo >Assignee: Jay Guo > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5250) Move 3rdparty/libprocess/3rdparty/* to 3rdparty/
[ https://issues.apache.org/jira/browse/MESOS-5250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5250: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Move 3rdparty/libprocess/3rdparty/* to 3rdparty/ > > > Key: MESOS-5250 > URL: https://issues.apache.org/jira/browse/MESOS-5250 > Project: Mesos > Issue Type: Task >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5174) Update the balloon-framework to run on test clusters
[
https://issues.apache.org/jira/browse/MESOS-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-5174:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Update the balloon-framework to run on test clusters
>
>
> Key: MESOS-5174
> URL: https://issues.apache.org/jira/browse/MESOS-5174
> Project: Mesos
> Issue Type: Improvement
> Components: framework, technical debt
>Reporter: Joseph Wu
>Assignee: Joseph Wu
> Labels: mesosphere, tech-debt
>
> There are a couple of problems with the balloon framework that prevent it
> from being deployed (easily) on an actual cluster:
> * The framework accepts 100% of memory in an offer. This means the expected
> behavior (finish or OOM) is dependent on the offer size.
> * The framework assumes the {{balloon-executor}} binary is available on each
> agent. This is generally only true in the build environment or in
> single-agent test environments.
> * The framework does not specify CPUs with the executor. This is required by
> many isolators.
> * The executor's {{TASK_FINISHED}} logic path was untested and is flaky.
> * The framework has no metrics.
> * The framework only launches a single task and then exits. With this
> behavior, we can't have useful metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5172) Registry puller cannot fetch blobs correctly from some private repos.
[ https://issues.apache.org/jira/browse/MESOS-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5172: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Registry puller cannot fetch blobs correctly from some private repos. > - > > Key: MESOS-5172 > URL: https://issues.apache.org/jira/browse/MESOS-5172 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Gilbert Song >Assignee: Gilbert Song > Labels: containerizer, mesosphere > > When the registry puller is pulling a private repository from some private > registry (e.g., quay.io), errors may occur when fetching blobs, at which > point fetching the manifest of the repo is finished correctly. The error > message is `Unexpected HTTP response '400 Bad Request' when trying to > download the blob`. This may arise from the logic of fetching blobs, or > incorrect format of uri when requesting blobs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4544) Propose design doc for agent partitioning behavior
[ https://issues.apache.org/jira/browse/MESOS-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4544: - Sprint: Mesosphere Sprint 28, Mesosphere Sprint 29, Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 28, Mesosphere Sprint 29, Mesosphere Sprint 30, Mesosphere Sprint 31, Mesosphere Sprint 32, Mesosphere Sprint 33) > Propose design doc for agent partitioning behavior > -- > > Key: MESOS-4544 > URL: https://issues.apache.org/jira/browse/MESOS-4544 > Project: Mesos > Issue Type: Task > Components: general >Reporter: Neil Conway >Assignee: Neil Conway > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5167) Add tests for `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5167: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add tests for `network/cni` isolator > > > Key: MESOS-5167 > URL: https://issues.apache.org/jira/browse/MESOS-5167 > Project: Mesos > Issue Type: Task > Components: test >Reporter: Qian Zhang >Assignee: Qian Zhang > Labels: mesosphere > > We need to add tests to verify the functionality of `network/cni` isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4785) Reorganize ACL subject/object descriptions
[ https://issues.apache.org/jira/browse/MESOS-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4785: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Reorganize ACL subject/object descriptions > -- > > Key: MESOS-4785 > URL: https://issues.apache.org/jira/browse/MESOS-4785 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Greg Mann >Assignee: Alexander Rojas > Labels: documentation, mesosphere, security > Fix For: 0.29.0 > > > The authorization documentation would benefit from a reorganization of the > ACL subject/object descriptions. Instead of simple lists of the available > subjects and objects, it would be nice to see a table showing which subject > and object is used with each action. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3214) Replace boost foreach with range-based for
[
https://issues.apache.org/jira/browse/MESOS-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-3214:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Replace boost foreach with range-based for
> --
>
> Key: MESOS-3214
> URL: https://issues.apache.org/jira/browse/MESOS-3214
> Project: Mesos
> Issue Type: Task
> Components: stout
>Reporter: Michael Park
>Assignee: Michael Park
> Labels: mesosphere
>
> It's desirable to replace the boost {{foreach}} macro with the C++11
> range-based {{for}}. This will help avoid some of the pitfalls of boost
> {{foreach}} such as dealing with types with commas in them, as well as
> improving compiler diagnostics by avoiding the macro expansion.
> One way to accomplish this is to replace the existing {{foreach (const Elem&
> elem, container)}} pattern with {{for (const Elem& elem : container)}}. We
> could support {{foreachkey}} and {{foreachvalue}} semantics via adaptors
> {{keys}} and {{values}} which would be used like this: {{for (const Key& key
> : keys(container))}}, {{for (const Value& value : values(container))}}. This
> leaves {{foreachpair}} which cannot be used with {{for}}. I think it would be
> desirable to support {{foreachpair}} for cases where the implicit unpacking
> is useful.
> Another approach is to keep {{foreach}}, {{foreachpair}}, {{foreachkey}} and
> {{foreachvalue}}, but simply implement them based on range-based {{for}}. For
> example, {{#define foreach(elem, container) for (elem : container)}}. While
> the consistency in the names is desirable, but unnecessary indirection of the
> macro definition is not.
> It's unclear to me which approach we would favor in Mesos, so please share
> your thoughts and preferences.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5173) Allow master/agent to take multiple --modules flags
[ https://issues.apache.org/jira/browse/MESOS-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5173: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Allow master/agent to take multiple --modules flags > --- > > Key: MESOS-5173 > URL: https://issues.apache.org/jira/browse/MESOS-5173 > Project: Mesos > Issue Type: Task >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > Fix For: 0.29.0 > > > When loading multiple modules into master/agent, one has to merge all module > metadata (library name, module name, parameters, etc.) into a single json > file which is then passed on to the --modules flag. This quickly becomes > cumbersome especially if the modules are coming from different > vendors/developers. > An alternate would be to allow multiple invocations of --modules flag that > can then be passed on to the module manager. That way, each flag corresponds > to just one module library and modules from that library. > Another approach is to create a new flag (e.g., --modules-dir) that contains > a path to a directory that would contain multiple json files. One can think > of it as an analogous to systemd units. The operator that drops a new file > into this directory and the file would automatically be picked up by the > master/agent module manager. Further, the naming scheme can also be inherited > to prefix the filename with an "NN_" to signify oad order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5153) Sandboxes contents should be protected from unauthorized users
[ https://issues.apache.org/jira/browse/MESOS-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5153: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Sandboxes contents should be protected from unauthorized users > -- > > Key: MESOS-5153 > URL: https://issues.apache.org/jira/browse/MESOS-5153 > Project: Mesos > Issue Type: Bug > Components: security, slave >Reporter: Alexander Rojas >Assignee: Alexander Rojas > Labels: mesosphere, security > Fix For: 0.29.0 > > > MESOS-4956 introduced authentication support for the sandboxes. However, > authentication can only go as far as to tell whether an user is known to > mesos or not. An extra additional step is necessary to verify whether the > known user is allowed to executed the requested operation on the sandbox > (browse, read, download, debug). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5221) Add Documentation for Nvidia GPU support
[ https://issues.apache.org/jira/browse/MESOS-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5221: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Add Documentation for Nvidia GPU support > > > Key: MESOS-5221 > URL: https://issues.apache.org/jira/browse/MESOS-5221 > Project: Mesos > Issue Type: Documentation >Reporter: Kevin Klues >Assignee: Kevin Klues >Priority: Minor > > https://reviews.apache.org/r/46220/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4386) Deprecate 'authenticate' master flag in favor of 'authenticate_frameworks' flag
[ https://issues.apache.org/jira/browse/MESOS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4386: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Deprecate 'authenticate' master flag in favor of 'authenticate_frameworks' > flag > --- > > Key: MESOS-4386 > URL: https://issues.apache.org/jira/browse/MESOS-4386 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: mesosphere, newbie, security > > To be consistent with `authenticate_slaves` and `authenticate_http` flags, we > should rename `authenticate` to `authenticate_frameworks` flag. > This should be done via deprecation cycle. > 1) Release X supports both `authenticate` and `authenticate_frameworks` flags > 2) Release X + n supports only `authenticate_frameworks` flag. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5155) Consolidate authorization actions for quota.
[
https://issues.apache.org/jira/browse/MESOS-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-5155:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Consolidate authorization actions for quota.
>
>
> Key: MESOS-5155
> URL: https://issues.apache.org/jira/browse/MESOS-5155
> Project: Mesos
> Issue Type: Improvement
>Reporter: Alexander Rukletsov
>Assignee: Zhitao Li
> Labels: mesosphere
>
> We should have just a single authz action: {{UPDATE_QUOTA_WITH_ROLE}}. It was
> a mistake in retrospect to introduce multiple actions.
> Actions that are not symmetrical are register/teardown and dynamic
> reservations. The way they are implemented in this way is because entities
> that do one action differ from entities that do the other. For example,
> register framework is issued by a framework, teardown by an operator. What is
> a good way to identify a framework? A role it runs in, which may be different
> each launch and makes no sense in multi-role frameworks setup or better a
> sort of a group id, which is its principal. For dynamic reservations and
> persistent volumes, they can be both issued by frameworks and operators,
> hence similar reasoning applies.
> Now, quota is associated with a role and set only by operators. Do we need to
> care about principals that set it? Not that much.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5139) ProvisionerDockerLocalStoreTest.LocalStoreTestWithTar is flaky
[
https://issues.apache.org/jira/browse/MESOS-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-5139:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> ProvisionerDockerLocalStoreTest.LocalStoreTestWithTar is flaky
> --
>
> Key: MESOS-5139
> URL: https://issues.apache.org/jira/browse/MESOS-5139
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.28.0
> Environment: Ubuntu14.04
>Reporter: Vinod Kone
>Assignee: Gilbert Song
> Labels: mesosphere
>
> Found this on ASF CI while testing 0.28.1-rc2
> {code}
> [ RUN ] ProvisionerDockerLocalStoreTest.LocalStoreTestWithTar
> E0406 18:29:30.870481 520 shell.hpp:93] Command 'hadoop version 2>&1'
> failed; this is the output:
> sh: 1: hadoop: not found
> E0406 18:29:30.870576 520 fetcher.cpp:59] Failed to create URI fetcher
> plugin 'hadoop': Failed to create HDFS client: Failed to execute 'hadoop
> version 2>&1'; the command was either not found or exited with a non-zero
> exit status: 127
> I0406 18:29:30.871052 520 local_puller.cpp:90] Creating local puller with
> docker registry '/tmp/3l8ZBv/images'
> I0406 18:29:30.873325 539 metadata_manager.cpp:159] Looking for image 'abc'
> I0406 18:29:30.874438 539 local_puller.cpp:142] Untarring image 'abc' from
> '/tmp/3l8ZBv/images/abc.tar' to '/tmp/3l8ZBv/store/staging/5tw8bD'
> I0406 18:29:30.901916 547 local_puller.cpp:162] The repositories JSON file
> for image 'abc' is '{"abc":{"latest":"456"}}'
> I0406 18:29:30.902304 547 local_puller.cpp:290] Extracting layer tar ball
> '/tmp/3l8ZBv/store/staging/5tw8bD/123/layer.tar to rootfs
> '/tmp/3l8ZBv/store/staging/5tw8bD/123/rootfs'
> I0406 18:29:30.909144 547 local_puller.cpp:290] Extracting layer tar ball
> '/tmp/3l8ZBv/store/staging/5tw8bD/456/layer.tar to rootfs
> '/tmp/3l8ZBv/store/staging/5tw8bD/456/rootfs'
> ../../src/tests/containerizer/provisioner_docker_tests.cpp:183: Failure
> (imageInfo).failure(): Collect failed: Subprocess 'tar, tar, -x, -f,
> /tmp/3l8ZBv/store/staging/5tw8bD/456/layer.tar, -C,
> /tmp/3l8ZBv/store/staging/5tw8bD/456/rootfs' failed: tar: This does not look
> like a tar archive
> tar: Exiting with failure status due to previous errors
> [ FAILED ] ProvisionerDockerLocalStoreTest.LocalStoreTestWithTar (243 ms)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5142) Add agent flags for HTTP authorization
[
https://issues.apache.org/jira/browse/MESOS-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-5142:
-
Sprint: Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34
(was: Mesosphere Sprint 32, Mesosphere Sprint 33)
> Add agent flags for HTTP authorization
> --
>
> Key: MESOS-5142
> URL: https://issues.apache.org/jira/browse/MESOS-5142
> Project: Mesos
> Issue Type: Bug
> Components: security, slave
>Reporter: Jan Schlicht
>Assignee: Jan Schlicht
> Labels: mesosphere, security
> Fix For: 0.29.0
>
>
> Flags should be added to the agent to:
> 1. Enable authorization ({{--authorizers}})
> 2. Provide ACLs ({{--acls}})
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5168: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3558) Implement HTTPCommandExecutor that uses the Executor Library
[
https://issues.apache.org/jira/browse/MESOS-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-3558:
-
Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint
33)
> Implement HTTPCommandExecutor that uses the Executor Library
> --
>
> Key: MESOS-3558
> URL: https://issues.apache.org/jira/browse/MESOS-3558
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Qian Zhang
> Labels: mesosphere
>
> Instead of using the {{MesosExecutorDriver}} , we should make the
> {{CommandExecutor}} in {{src/launcher/executor.cpp}} use the new Executor
> HTTP Library that we create in {{MESOS-3550}}.
> This would act as a good validation of the {{HTTP API}} implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-5064) Remove default value for the agent `work_dir`
[
https://issues.apache.org/jira/browse/MESOS-5064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Harutyunyan updated MESOS-5064:
-
Sprint: Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34
(was: Mesosphere Sprint 32, Mesosphere Sprint 33)
> Remove default value for the agent `work_dir`
> -
>
> Key: MESOS-5064
> URL: https://issues.apache.org/jira/browse/MESOS-5064
> Project: Mesos
> Issue Type: Bug
>Reporter: Artem Harutyunyan
>Assignee: Greg Mann
>
> Following a crash report from the user we need to be more explicit about the
> dangers of using {{/tmp}} as agent {{work_dir}}. In addition, we can remove
> the default value for the {{\-\-work_dir}} flag, forcing users to explicitly
> set the work directory for the agent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-4690) Reorganize 3rdparty directory
[ https://issues.apache.org/jira/browse/MESOS-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4690: - Sprint: Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 33) > Reorganize 3rdparty directory > - > > Key: MESOS-4690 > URL: https://issues.apache.org/jira/browse/MESOS-4690 > Project: Mesos > Issue Type: Epic > Components: build, libprocess, stout >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > > This issues is currently being discussed in the dev mailing list: > http://www.mail-archive.com/dev@mesos.apache.org/msg34349.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5051) Create helpers for manipulating Linux capabilities.
[ https://issues.apache.org/jira/browse/MESOS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-5051: - Sprint: Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 32, Mesosphere Sprint 33) > Create helpers for manipulating Linux capabilities. > --- > > Key: MESOS-5051 > URL: https://issues.apache.org/jira/browse/MESOS-5051 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jojy Varghese > Labels: mesosphere > > These helpers can either based on some existing library (e.g. libcap), or use > system calls directly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4766) Improve allocator performance.
[ https://issues.apache.org/jira/browse/MESOS-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-4766: - Sprint: Mesosphere Sprint 32, Mesosphere Sprint 33, Mesosphere Sprint 34 (was: Mesosphere Sprint 32, Mesosphere Sprint 33) > Improve allocator performance. > -- > > Key: MESOS-4766 > URL: https://issues.apache.org/jira/browse/MESOS-4766 > Project: Mesos > Issue Type: Epic > Components: allocation >Reporter: Benjamin Mahler >Assignee: Michael Park >Priority: Critical > > This is an epic to track the various tickets around improving the performance > of the allocator, including the following: > * Preventing un-necessary backup of the allocator. > * Reducing the cost of allocations and allocator state updates. > * Improving performance of the DRF sorter. > * More benchmarking to simulate scenarios with performance issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
[
https://issues.apache.org/jira/browse/MESOS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Bannier updated MESOS-5293:
Description:
The way endpoints are routed is inconsistent between master and agent code
which makes adding or extending handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to access master state from an endpoint handler. The handler is a (probably
non-static) member function of the master's member variable {{http}} which
outlives the callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} by value into the
callback. Since the callback is currently treated like a value and might e.g.,
be freely copied around we are only guaranteed that {{http}} lives long enough
for the handler (here {{Http::executor}}) to return. In particular, since
endpoint handlers return a {{Future}} there is no guarantee that the
used {{http}} lives long enough to be still valid once a conventional (e.g.,
non-static member) continuation is executed.
Both models have their merit:
* capturing a pointer to the master simplifies reasoning about lifetimes,
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master pattern.
was:
The way endpoints are routed is inconsistent between master and agent code
which makes adding or extending handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to access master state from an endpoint handler. The handler is a (probably
non-static) member function of the master's member variable {{http}} which
outlives the callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing a pointer to the master simplifies reasoning about lifetimes,
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the
[jira] [Updated] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
[
https://issues.apache.org/jira/browse/MESOS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Bannier updated MESOS-5293:
Description:
The way endpoints are routed is inconsistent between master and agent code
which makes adding or extending handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to access master state from an endpoint handler. The handler is a (probably
non-static) member function of the master's member variable {{http}} which
outlives the callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing a pointer to the master simplifies reasoning about lifetimes,
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master pattern.
was:
The way endpoints in routed is inconsistent between master and agent code which
makes added new handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to access master state from an endpoint handler. The handler is a (probably
non-static) member function of the master's member variable {{http}} which
outlives the callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing a pointer to the master simplifies reasoning about lifetimes,
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master
[jira] [Updated] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
[
https://issues.apache.org/jira/browse/MESOS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Bannier updated MESOS-5293:
Description:
The way endpoints in routed is inconsistent between master and agent code which
makes added new handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to access master state from an endpoint handler. The handler is a (probably
non-static) member function of the master's member variable {{http}} which
outlives the callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing a pointer to the master simplifies reasoning about lifetimes,
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master pattern.
was:
The way endpoints in routed is inconsistent between master and agent code which
makes added new handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to
access master state from an endpoint handler. The handler is a (probably
non-static)
member function of the master's member variable {{http}} which outlives the
callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing {{this}} simplifies reasoning about lifetimes
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master pattern.
> Endpoint handlers for
[jira] [Created] (MESOS-5293) Endpoint handlers for master and agent need to be implemented surprisingly differently
Benjamin Bannier created MESOS-5293:
---
Summary: Endpoint handlers for master and agent need to be
implemented surprisingly differently
Key: MESOS-5293
URL: https://issues.apache.org/jira/browse/MESOS-5293
Project: Mesos
Issue Type: Bug
Components: master, slave
Reporter: Benjamin Bannier
The way endpoints in routed is inconsistent between master and agent code which
makes added new handlers error prone.
In the master we route like this:
{code}
// Setup HTTP routes.
route("/api/v1/scheduler",
DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM,
Http::SCHEDULER_HELP(),
[this](const process::http::Request& request,
const Option& principal) {
Http::log(request);
return http.scheduler(request, principal);
});
{code}
We capture a pointer to the current {{Master}} in the callback which allows us
to
access master state from an endpoint handler. The handler is a (probably
non-static)
member function of the master's member variable {{http}} which outlives the
callback.
Routing in the agent looks like this:
{code}
// Setup HTTP routes.
Http http = Http(this);
route("/api/v1/executor",
Http::EXECUTOR_HELP(),
[http](const process::http::Request& request) {
Http::log(request);
return http.executor(request);
});
{code}
In contrast to the master code we here copy a {{Http}} into the callback. Since
the callback is currently treated like a value and might e.g., be freely copied
around we are only guaranteed that it lives long enough for the handler (here
{{Http::executor}}) to return. In particular, since endpoint handlers return a
{{Future}} there is no guarantee that the used {{http}} lives long
enough to be still valid once a conventional (e.g., non-static member)
continuation is executed.
Both models have their merit:
* capturing {{this}} simplifies reasoning about lifetimes
* capturing just a {{Http}} with very short lifetime minimizes interactions
among (e.g., concurrent) invocations of endpoint handlers.
This great inconsistency comes with a cost though, as employing patterns
borrowed from master endpoint handlers in agent code will lead to potentially
subtle bugs where a developer assuming that {{http}} would outlive a handler's
execution might introduce code invoking member functions of already destructed
variables. This is especially likely in code employing multiple layers of
{{delay}} or {{defer}} for which compilers seem unable to detect lifetime
problems and emit diagnostics.
It would be great if we could to use just one of the patterns to minimize
confusion, e.g., the more straight-forward master pattern.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3319) Mesos will not build when configured with gperftools enabled
[
https://issues.apache.org/jira/browse/MESOS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260119#comment-15260119
]
Kapil Arya edited comment on MESOS-3319 at 4/27/16 1:15 PM:
{code}
commit dc8b9a4abff74d1a2e33039836682bdfa7188f87
Author: Greg Mann
Date: Tue Apr 26 10:43:50 2016 -0400
Added the '--enable-perftools' flag to configure docs.
Review: https://reviews.apache.org/r/46647/
commit 0956ce89bad62657e88affe233a6693632a5bc67
Author: Greg Mann
Date: Tue Apr 26 10:43:37 2016 -0400
Updated gperftools to version 2.5 (Mesos).
Review: https://reviews.apache.org/r/46462/
commit 5aec0377bdaba4aa72debdf37aeb66960e5192d1
Author: Greg Mann
Date: Tue Apr 26 10:43:27 2016 -0400
Updated gperftools to version 2.5 (libprocess).
Review: https://reviews.apache.org/r/46461/
commit 50a22f3d688faec95937090d3e6d75ae641dc940
Author: Greg Mann
Date: Tue Apr 26 10:43:33 2016 -0400
Fixed the build when perftools is enabled.
Review: https://reviews.apache.org/r/46643/
{code}
was (Author: karya):
{code}
commit 0956ce89bad62657e88affe233a6693632a5bc67
Author: Greg Mann
Date: Tue Apr 26 10:43:37 2016 -0400
Updated gperftools to version 2.5 (Mesos).
Review: https://reviews.apache.org/r/46462/
commit 5aec0377bdaba4aa72debdf37aeb66960e5192d1
Author: Greg Mann
Date: Tue Apr 26 10:43:27 2016 -0400
Updated gperftools to version 2.5 (libprocess).
Review: https://reviews.apache.org/r/46461/
commit 50a22f3d688faec95937090d3e6d75ae641dc940
Author: Greg Mann
Date: Tue Apr 26 10:43:33 2016 -0400
Fixed the build when perftools is enabled.
Review: https://reviews.apache.org/r/46643/
{code}
> Mesos will not build when configured with gperftools enabled
>
>
> Key: MESOS-3319
> URL: https://issues.apache.org/jira/browse/MESOS-3319
> Project: Mesos
> Issue Type: Bug
>Reporter: Greg Mann
>Assignee: Greg Mann
> Labels: build
> Fix For: 0.29.0
>
>
> Mesos configured with {{--enable-perftools}} currently will not build on OSX
> 10.10.4 or Ubuntu 14.04, possibly because the bundled gperftools-2.0 is not
> current. The stable release is now 2.4, which builds successfully on both of
> these platforms.
> This issue is resolved when Mesos will build successfully out of the box with
> gperftools enabled. After this ticket is resolved, the libprocess profiler
> should be tested to confirm that it still works and if not, it should be
> fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3319) Mesos will not build when configured with gperftools enabled
[
https://issues.apache.org/jira/browse/MESOS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260119#comment-15260119
]
Kapil Arya commented on MESOS-3319:
---
{code}
commit 0956ce89bad62657e88affe233a6693632a5bc67
Author: Greg Mann
Date: Tue Apr 26 10:43:37 2016 -0400
Updated gperftools to version 2.5 (Mesos).
Review: https://reviews.apache.org/r/46462/
commit 5aec0377bdaba4aa72debdf37aeb66960e5192d1
Author: Greg Mann
Date: Tue Apr 26 10:43:27 2016 -0400
Updated gperftools to version 2.5 (libprocess).
Review: https://reviews.apache.org/r/46461/
commit 50a22f3d688faec95937090d3e6d75ae641dc940
Author: Greg Mann
Date: Tue Apr 26 10:43:33 2016 -0400
Fixed the build when perftools is enabled.
Review: https://reviews.apache.org/r/46643/
{code}
> Mesos will not build when configured with gperftools enabled
>
>
> Key: MESOS-3319
> URL: https://issues.apache.org/jira/browse/MESOS-3319
> Project: Mesos
> Issue Type: Bug
>Reporter: Greg Mann
>Assignee: Greg Mann
> Labels: build
> Fix For: 0.29.0
>
>
> Mesos configured with {{--enable-perftools}} currently will not build on OSX
> 10.10.4 or Ubuntu 14.04, possibly because the bundled gperftools-2.0 is not
> current. The stable release is now 2.4, which builds successfully on both of
> these platforms.
> This issue is resolved when Mesos will build successfully out of the box with
> gperftools enabled. After this ticket is resolved, the libprocess profiler
> should be tested to confirm that it still works and if not, it should be
> fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4951) Enable actors to pass an authentication realm to libprocess
[
https://issues.apache.org/jira/browse/MESOS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260109#comment-15260109
]
Kapil Arya commented on MESOS-4951:
---
{code}
commit 23bff549128c50586d34cda110e6f02021242973
Author: Greg Mann
Date: Tue Apr 26 10:42:42 2016 -0400
Added a realm parameter to 'process::initialize' (Mesos).
In order to enable authentication on libprocess-level
HTTP endpoints, this patch adds code to the master and
agent's main.cpp file which makes use of the new
`authenticationRealm` argument to `process::initialize`
which allows the authentication realm of such endpoints
to be set when libprocess is initialized. The argument is
added to libprocess initialization in the tests as well.
Review: https://reviews.apache.org/r/46255/
commit 3d781197873c8f388a9ed9152bb1b3686d32
Author: Greg Mann
Date: Tue Apr 26 10:42:25 2016 -0400
Added a realm parameter to `process::initialize` (libprocess).
In order to enable authentication on libprocess-level
HTTP endpoints, this patch adds a parameter to
process::initialize which allows the authentication
realm of such endpoints to be set when libprocess is
initialized.
Review: https://reviews.apache.org/r/46254/
{code}
> Enable actors to pass an authentication realm to libprocess
> ---
>
> Key: MESOS-4951
> URL: https://issues.apache.org/jira/browse/MESOS-4951
> Project: Mesos
> Issue Type: Improvement
> Components: libprocess, slave
>Reporter: Greg Mann
>Assignee: Greg Mann
> Labels: authentication, http, mesosphere, security
> Fix For: 0.29.0
>
>
> To prepare for MESOS-4902, the Mesos master and agent need a way to pass the
> desired authentication realm to libprocess. Since some endpoints (like
> {{/profiler/*}}) get installed in libprocess, the master/agent should be able
> to specify during initialization what authentication realm the
> libprocess-level endpoints will be authenticated under.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5263) pivot_root is not available on ARM
[
https://issues.apache.org/jira/browse/MESOS-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259893#comment-15259893
]
Tomasz Janiszewski commented on MESOS-5263:
---
Cleanup review: https://reviews.apache.org/r/46730/
> pivot_root is not available on ARM
> --
>
> Key: MESOS-5263
> URL: https://issues.apache.org/jira/browse/MESOS-5263
> Project: Mesos
> Issue Type: Bug
>Reporter: Tomasz Janiszewski
>Assignee: Tomasz Janiszewski
> Fix For: 0.29.0
>
>
> When compile on ARM, it will through error.
> The current code logic in src/linux/fs.cpp is:
> {code}
> #ifdef __NR_pivot_root
> int ret = ::syscall(__NR_pivot_root, newRoot.c_str(), putOld.c_str());
> #elif __x86_64__
> // A workaround for systems that have an old glib but have a new
> // kernel. The magic number '155' is the syscall number for
> // 'pivot_root' on the x86_64 architecture, see
> // arch/x86/syscalls/syscall_64.tbl
> int ret = ::syscall(155, newRoot.c_str(), putOld.c_str());
> #elif __powerpc__ || __ppc__ || __powerpc64__ || __ppc64__
> // A workaround for powerpc. The magic number '203' is the syscall
> // number for 'pivot_root' on the powerpc architecture, see
> // https://w3challs.com/syscalls/?arch=powerpc_64
> int ret = ::syscall(203, newRoot.c_str(), putOld.c_str());
> #else
> #error "pivot_root is not available"
> #endif
> {code}
> Possible sollution is to add `unistd.h` header
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4902) Add authentication to libprocess endpoints
[
https://issues.apache.org/jira/browse/MESOS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259873#comment-15259873
]
Adam B commented on MESOS-4902:
---
Reopened because none of the RB patches are marked submitted.
Please also copy the commit messages into the ticket when you (or preferably
your shepherd) resolve it.
> Add authentication to libprocess endpoints
> --
>
> Key: MESOS-4902
> URL: https://issues.apache.org/jira/browse/MESOS-4902
> Project: Mesos
> Issue Type: Improvement
> Components: HTTP API
>Reporter: Greg Mann
>Assignee: Greg Mann
> Labels: authentication, http, mesosphere, security
> Fix For: 0.29.0
>
>
> In addition to the endpoints addressed by MESOS-4850 and MESOS-5152, the
> following endpoints would also benefit from HTTP authentication:
> * {{/profiler/*}}
> * {{/logging/toggle}}
> * {{/metrics/snapshot}}
> Adding HTTP authentication to these endpoints is a bit more complicated
> because they are defined at the libprocess level.
> While working on MESOS-4850, it became apparent that since our tests use the
> same instance of libprocess for both master and agent, different default
> authentication realms must be used for master/agent so that HTTP
> authentication can be independently enabled/disabled for each.
> We should establish a mechanism for making an endpoint authenticated that
> allows us to:
> 1) Install an endpoint like {{/files}}, whose code is shared by the master
> and agent, with different authentication realms for the master and agent
> 2) Avoid hard-coding a default authentication realm into libprocess, to
> permit the use of different authentication realms for the master and agent
> and to keep application-level concerns from leaking into libprocess
> Another option would be to use a single default authentication realm and
> always enable or disable HTTP authentication for *both* the master and agent
> in tests. However, this wouldn't allow us to test scenarios where HTTP
> authentication is enabled on one but disabled on the other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Ma updated MESOS-1739: Comment: was deleted (was: Agree to send notification to the the frameworks. For the executors/tasks, will we kill them if slave's attributes changed? I think it's better to keep them running: 1. update slave's attributes, 2. notify frameworks & reconcile tasks, 3. slave killed tasks if necessary; no impact to other executor/tasks.) > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259825#comment-15259825 ] Klaus Ma commented on MESOS-1739: - Agree to send notification to the the frameworks. For the executors/tasks, will we kill them if slave's attributes changed? I think it's better to keep them running: 1. update slave's attributes, 2. notify frameworks & reconcile tasks, 3. slave killed tasks if necessary; no impact to other executor/tasks. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3779) Slave/Agent Rename Phase I - Update terms in Web UI.
[ https://issues.apache.org/jira/browse/MESOS-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhou xing updated MESOS-3779: - Summary: Slave/Agent Rename Phase I - Update terms in Web UI. (was: Slave/Agent Rename Phase I - Update webui) > Slave/Agent Rename Phase I - Update terms in Web UI. > > > Key: MESOS-3779 > URL: https://issues.apache.org/jira/browse/MESOS-3779 > Project: Mesos > Issue Type: Task >Reporter: Diana Arroyo >Assignee: zhou xing > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3778) Slave/Agent Rename Phase I - Add duplicate HTTP endpoints
[ https://issues.apache.org/jira/browse/MESOS-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259773#comment-15259773 ] zhou xing commented on MESOS-3778: -- [~vi...@twitter.com], so do we need to close this one? or transfer the owner to you? > Slave/Agent Rename Phase I - Add duplicate HTTP endpoints > - > > Key: MESOS-3778 > URL: https://issues.apache.org/jira/browse/MESOS-3778 > Project: Mesos > Issue Type: Task >Reporter: Diana Arroyo >Assignee: zhou xing > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3783) Replace Master/Slave Terminology Phase I - Update documentation
[ https://issues.apache.org/jira/browse/MESOS-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259771#comment-15259771 ] Jay Guo commented on MESOS-3783: As I start replacing 'slave' in docs, I found it really awkward to leave them in an inconsistent state since some docs are blocked by actual code changes. Firstly, I don't think we should have both terms mixed in one single doc Secondly, lot of docs refer to flags, metrics, endpoints, public interfaces. If we continue with the replacement, we end up with significant part of docs using 'slave', and part of them using 'agent'. Therefore, I would suggest to do this at last, or at least after the majority of code change is done. [~vinodkone] > Replace Master/Slave Terminology Phase I - Update documentation > > > Key: MESOS-3783 > URL: https://issues.apache.org/jira/browse/MESOS-3783 > Project: Mesos > Issue Type: Task >Reporter: Diana Arroyo >Assignee: Jay Guo > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3784) Replace Master/Slave Terminology Phase I - Update mesos-cli
[ https://issues.apache.org/jira/browse/MESOS-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Guo reassigned MESOS-3784: -- Assignee: Jay Guo > Replace Master/Slave Terminology Phase I - Update mesos-cli > > > Key: MESOS-3784 > URL: https://issues.apache.org/jira/browse/MESOS-3784 > Project: Mesos > Issue Type: Task >Reporter: Diana Arroyo >Assignee: Jay Guo > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3784) Replace Master/Slave Terminology Phase I - Update mesos-cli
[
https://issues.apache.org/jira/browse/MESOS-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259728#comment-15259728
]
Jay Guo commented on MESOS-3784:
I took a look at this ticket and here are some findings:
1. root command {{mesos}} collects scripts whose name match {{mesos-*}} in same
dir as the subcommands. As long as we keep mesos-slave binary and its wrapper
script, {{slave}} appears in the subcommands, unless we explicitly filter it
out in {{mesos}} executable. I would suggest to add deprecation warning when
using mesos-slave.
2. Subcommand's usage info are basically flags descriptions, which are deeply
coupled with https://issues.apache.org/jira/browse/MESOS-3781 I suggest we
solve them there.
3. Python utils _mesos-cat_, _mesos-ps_, _mesos-tail_ and _mesos-scp_ do
contain few hardcoded "slave". Although I wonder whether we still keep these
utils? In pypi page, it's stated that mesos.cli is deprecated in favor of DCOS
CLI. https://pypi.python.org/pypi/mesos.cli For one thing, these utils assume
a zookeeper setup and resolve master from them, therefore not useful in a
standalone setup. Secondly, there are some bugs in the util, such as
referencing variables in {{finally}} which is assigned in {{try}} without a
default value. Is there someone maintaining these scripts? Should we fix them
systematically?
[~vinodkone] ideas?
> Replace Master/Slave Terminology Phase I - Update mesos-cli
>
>
> Key: MESOS-3784
> URL: https://issues.apache.org/jira/browse/MESOS-3784
> Project: Mesos
> Issue Type: Task
>Reporter: Diana Arroyo
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3224) Create a Mesos Contributor Newbie Guide
[ https://issues.apache.org/jira/browse/MESOS-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259699#comment-15259699 ] haosdent commented on MESOS-3224: - Thank you very much! > Create a Mesos Contributor Newbie Guide > --- > > Key: MESOS-3224 > URL: https://issues.apache.org/jira/browse/MESOS-3224 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Timothy Chen >Assignee: Diana Arroyo > > Currently the website doesn't have a helpful guide for community users to > know how to start learning to contribute to Mesos, understand the concepts > and lower the barrier to get involved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5292) Website references non-existing file NewbieContributionOverview.jpg
[
https://issues.apache.org/jira/browse/MESOS-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259693#comment-15259693
]
Benjamin Bannier commented on MESOS-5292:
-
[~darroyo] Could you have a look if you still have access to that file?
> Website references non-existing file NewbieContributionOverview.jpg
> ---
>
> Key: MESOS-5292
> URL: https://issues.apache.org/jira/browse/MESOS-5292
> Project: Mesos
> Issue Type: Bug
> Components: project website
>Reporter: Benjamin Bannier
> Labels: site
>
> The website references the non-existing file
> {{NewbieContributionOverview.jpg}} in {{docs/newbie-guide.md}}. Looking at
> the commit adding this documentation it appears this file was never added to
> the repository. We should either provide the file or rewrite the section to
> work without needing to reference it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3224) Create a Mesos Contributor Newbie Guide
[ https://issues.apache.org/jira/browse/MESOS-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259697#comment-15259697 ] Benjamin Bannier commented on MESOS-3224: - [~haosd...@gmail.com], I filed MESOS-5292. > Create a Mesos Contributor Newbie Guide > --- > > Key: MESOS-3224 > URL: https://issues.apache.org/jira/browse/MESOS-3224 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Timothy Chen >Assignee: Diana Arroyo > > Currently the website doesn't have a helpful guide for community users to > know how to start learning to contribute to Mesos, understand the concepts > and lower the barrier to get involved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5292) Website references non-existing file NewbieContributionOverview.jpg
Benjamin Bannier created MESOS-5292:
---
Summary: Website references non-existing file
NewbieContributionOverview.jpg
Key: MESOS-5292
URL: https://issues.apache.org/jira/browse/MESOS-5292
Project: Mesos
Issue Type: Bug
Components: project website
Reporter: Benjamin Bannier
The website references the non-existing file {{NewbieContributionOverview.jpg}}
in {{docs/newbie-guide.md}}. Looking at the commit adding this documentation it
appears this file was never added to the repository. We should either provide
the file or rewrite the section to work without needing to reference it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5168) Benchmark overhead of authorization based filtering.
[ https://issues.apache.org/jira/browse/MESOS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259605#comment-15259605 ] Adam B commented on MESOS-5168: --- Although I'm glad that you wrote a benchmark framework, I wouldn't consider this task reviewable until you've run the benchmark and can tell me how much overhead you've added. > Benchmark overhead of authorization based filtering. > > > Key: MESOS-5168 > URL: https://issues.apache.org/jira/browse/MESOS-5168 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad > Labels: authorization, mesosphere, security > Fix For: 0.29.0 > > > When adding authorization based filtering as outlined in MESOS-4931 we need > to be careful especially for performance critical endpoints such as /state. > We should ensure via a benchmark that performance does not degreade below an > acceptable state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2222) Add ACLs for the maintenance HTTP endpoints.
[ https://issues.apache.org/jira/browse/MESOS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259603#comment-15259603 ] Adam B commented on MESOS-: --- Recommend we do coarse-grained authz using GET|POST_ENDPOINT_WITH_PATH, since it's hard to say who should be able to shut down which machines, except to say that only and administrator should do it (regardless of IP/hostname). Lower priority though, so dropping this out of the current Mesosphere sprint. > Add ACLs for the maintenance HTTP endpoints. > > > Key: MESOS- > URL: https://issues.apache.org/jira/browse/MESOS- > Project: Mesos > Issue Type: Task > Components: master, security >Affects Versions: 0.25.0 >Reporter: Benjamin Mahler > Labels: authorization, maintenance, mesosphere, security > > In order to authorize the HTTP endpoints for maintenance (to be added in > MESOS-2067), we will need to add an ACL definition for performing maintenance > operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
