[jira] [Assigned] (MESOS-7048) Remove adjustment code within Resources::apply.
[ https://issues.apache.org/jira/browse/MESOS-7048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Guo reassigned MESOS-7048: -- Assignee: Jay Guo > Remove adjustment code within Resources::apply. > --- > > Key: MESOS-7048 > URL: https://issues.apache.org/jira/browse/MESOS-7048 > Project: Mesos > Issue Type: Task > Components: technical debt >Reporter: Benjamin Mahler >Assignee: Jay Guo > > Currently, {{Resources::apply()}} will strip allocation info from operation's > resources in order to have operations apply correctly to unallocated > resources. To make this more explicit, we should move the stripping up into > the call sites that need it. We'll need a helper to do this. > As a result, the master and allocator will need to strip prior to applying > operations to the agent's total resources (which are stored as unallocated > resources). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7203) Add a '--require_http_authentication' flag
[ https://issues.apache.org/jira/browse/MESOS-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-7203: - Description: The current HTTP authentication implementation in Mesos makes it difficult to properly authorize some operations when authentication is not enabled. The {{UNRESERVE}} and {{DESTROY}} operations use a {{principal}} field stored in {{ReservationInfo}}/{{DiskInfo}} for authorization. This means that in order to authorize properly, the principal responsible for the reservation/volume must be available when the {{RESERVE}}/{{CREATE}} operation is performed. However, if HTTP authentication is not enabled, then operators are not able to provide a principal. In order to resolve this issue, a new {{\-\-require_http_authentication}} field could be added. This flag would complement the {{\-\-http_authenticators}} flag. The new behavior would be as follows: * If {{\-\-http_authenticators}} is set but {{\-\-require_http_authentication}} is not set, the authenticators would be loaded as specified, but unauthenticated requests would be permitted. In the case of an HTTP request containing an {{Authorization}} header, the header would be used to construct a {{Principal}} to be passed to the handlers. * If {{\-\-http_authenticators}} is set and {{\-\-require_http_authentication}} is also set, the {{Principal}} would be extracted and passed to handlers as before, but all requests without an authenticated principal would be rejected. was: The current HTTP authentication implementation in Mesos makes it difficult to properly authorize some operations when authentication is not enabled. The {{UNRESERVE}} and {{DESTROY}} operations use a {{principal}} field stored in {{ReservationInfo}}/{{DiskInfo}} for authorization. This means that in order to authorize properly, the principal responsible for the reservation/volume must be available when the {{RESERVE}}/{{CREATE}} operation is performed. However, if HTTP authentication is not enabled, then operators are not able to provide a principal. In order to resolve this issue, a new {{--require_http_authentication}} field could be added. This flag would complement the {{--http_authenticators}} flag. The new behavior would be as follows: * If {{--http_authenticators}} is set but {{--require_http_authentication}} is not set, the authenticators would be loaded as specified, but unauthenticated requests would be permitted. In the case of an HTTP request containing an {{Authorization}} header, the header would be used to construct a {{Principal}} to be passed to the handlers. * If {{--http_authenticators}} is set and {{--require_http_authentication}} is also set, the {{Principal}} would be extracted and passed to handlers as before, but all requests without an authenticated principal would be rejected. > Add a '--require_http_authentication' flag > -- > > Key: MESOS-7203 > URL: https://issues.apache.org/jira/browse/MESOS-7203 > Project: Mesos > Issue Type: Improvement > Components: security >Reporter: Greg Mann > Labels: authentication, http, mesosphere > > The current HTTP authentication implementation in Mesos makes it difficult to > properly authorize some operations when authentication is not enabled. The > {{UNRESERVE}} and {{DESTROY}} operations use a {{principal}} field stored in > {{ReservationInfo}}/{{DiskInfo}} for authorization. This means that in order > to authorize properly, the principal responsible for the reservation/volume > must be available when the {{RESERVE}}/{{CREATE}} operation is performed. > However, if HTTP authentication is not enabled, then operators are not able > to provide a principal. > In order to resolve this issue, a new {{\-\-require_http_authentication}} > field could be added. This flag would complement the > {{\-\-http_authenticators}} flag. The new behavior would be as follows: > * If {{\-\-http_authenticators}} is set but > {{\-\-require_http_authentication}} is not set, the authenticators would be > loaded as specified, but unauthenticated requests would be permitted. In the > case of an HTTP request containing an {{Authorization}} header, the header > would be used to construct a {{Principal}} to be passed to the handlers. > * If {{\-\-http_authenticators}} is set and > {{\-\-require_http_authentication}} is also set, the {{Principal}} would be > extracted and passed to handlers as before, but all requests without an > authenticated principal would be rejected. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7203) Add a '--require_http_authentication' flag
Greg Mann created MESOS-7203: Summary: Add a '--require_http_authentication' flag Key: MESOS-7203 URL: https://issues.apache.org/jira/browse/MESOS-7203 Project: Mesos Issue Type: Improvement Components: security Reporter: Greg Mann The current HTTP authentication implementation in Mesos makes it difficult to properly authorize some operations when authentication is not enabled. The {{UNRESERVE}} and {{DESTROY}} operations use a {{principal}} field stored in {{ReservationInfo}}/{{DiskInfo}} for authorization. This means that in order to authorize properly, the principal responsible for the reservation/volume must be available when the {{RESERVE}}/{{CREATE}} operation is performed. However, if HTTP authentication is not enabled, then operators are not able to provide a principal. In order to resolve this issue, a new {{--require_http_authentication}} field could be added. This flag would complement the {{--http_authenticators}} flag. The new behavior would be as follows: * If {{--http_authenticators}} is set but {{--require_http_authentication}} is not set, the authenticators would be loaded as specified, but unauthenticated requests would be permitted. In the case of an HTTP request containing an {{Authorization}} header, the header would be used to construct a {{Principal}} to be passed to the handlers. * If {{--http_authenticators}} is set and {{--require_http_authentication}} is also set, the {{Principal}} would be extracted and passed to handlers as before, but all requests without an authenticated principal would be rejected. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7202) Allow 'principal.value' to be NONE in master handlers
Greg Mann created MESOS-7202: Summary: Allow 'principal.value' to be NONE in master handlers Key: MESOS-7202 URL: https://issues.apache.org/jira/browse/MESOS-7202 Project: Mesos Issue Type: Improvement Reporter: Greg Mann The {{Principal}} type was recently introduced in libprocess to facilitate executor authentication (MESOS-6365). Currently, we do not allow {{Principal.value}} to be {{NONE}} in the master handler code for the following reasons: * The master's internal structures (i.e. {{frameworks.principals}}) associate each framework with a {{string principal}} * The {{ReservationInfo}} and {{DiskInfo}} messages store a {{string principal}} for the purpose of authorizing {{UNRESERVE}} and {{DESTROY}} operations We should migrate the master's internal structures, as well as the {{ReservationInfo}} and {{DiskInfo}} messages, to make use of an object similar to {{process::http::authentication::Principal}}. When implementing this for {{ReservationInfo}} and {{DiskInfo}}, we should consider including the principal in the Mesos internal message representations, while omitting the principal from the external user-facing messages. This would eliminate the need for the user to duplicate their principal in the contents of their {{RESERVE}}/{{CREATE}} request, when they must already supply it in the request's authorization header. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-6691) Enable SSL in Mesos builds
[ https://issues.apache.org/jira/browse/MESOS-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Wu updated MESOS-6691: - Description: (was: | https://reviews.apache.org/r/57260/ | A bit of refactoring | | https://reviews.apache.org/r/57261/ | Configuration flags | | https://reviews.apache.org/r/57262/ | SSL source files |) > Enable SSL in Mesos builds > -- > > Key: MESOS-6691 > URL: https://issues.apache.org/jira/browse/MESOS-6691 > Project: Mesos > Issue Type: Task > Components: agent >Reporter: Alex Clemmer >Assignee: Joseph Wu > Labels: microsoft, windows-mvp > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6691) Enable SSL in Mesos builds
[ https://issues.apache.org/jira/browse/MESOS-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893280#comment-15893280 ] Joseph Wu commented on MESOS-6691: -- | https://reviews.apache.org/r/57260/ | A bit of refactoring | | https://reviews.apache.org/r/57261/ | Configuration flags | | https://reviews.apache.org/r/57262/ | SSL source files | > Enable SSL in Mesos builds > -- > > Key: MESOS-6691 > URL: https://issues.apache.org/jira/browse/MESOS-6691 > Project: Mesos > Issue Type: Task > Components: agent >Reporter: Alex Clemmer >Assignee: Joseph Wu > Labels: microsoft, windows-mvp > > | https://reviews.apache.org/r/57260/ | A bit of refactoring | > | https://reviews.apache.org/r/57261/ | Configuration flags | > | https://reviews.apache.org/r/57262/ | SSL source files | -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7201) Improvements to maintenance primitives
Joseph Wu created MESOS-7201: Summary: Improvements to maintenance primitives Key: MESOS-7201 URL: https://issues.apache.org/jira/browse/MESOS-7201 Project: Mesos Issue Type: Epic Reporter: Joseph Wu This is a follow up epic to MESOS-1474 to capture further improvements for maintenance primitives. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893019#comment-15893019 ] Benjamin Bannier commented on MESOS-7197: - Yes, with the same clang version even an unoptimized build causes us to run into a {{CHECK}} failure with both test cases. \[1\] clang version 5.0.0 (http://llvm.org/git/clang 60de4ba003932b44c77702815e0485d77e55c9c3) (http://llvm.org/git/llvm 85f508cd9dba8a982471d98c4f649fb0d63f3451) Target: x86_64-apple-darwin16.4.0 Thread model: posix InstalledDir: /Users/bbannier/src/llvm/P/bin > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Assignee: Neil Conway >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7026) Update authorization / authorization-filtering to handle hierarchical roles.
[ https://issues.apache.org/jira/browse/MESOS-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier updated MESOS-7026: Shepherd: Adam B Sprint: Mesosphere Sprint 53 Story Points: 5 > Update authorization / authorization-filtering to handle hierarchical roles. > > > Key: MESOS-7026 > URL: https://issues.apache.org/jira/browse/MESOS-7026 > Project: Mesos > Issue Type: Task > Components: agent, HTTP API, master >Reporter: Benjamin Mahler >Assignee: Benjamin Bannier > > Authorization and endpoint filtering will need to be updated in order to > allow the authorization to be performed in a hierarchical manner (e.g. a user > can see all beneath /eng/* vs. a user can see all beneath /eng/frontend/*). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Issue Comment Deleted] (MESOS-6375) Support hierarchical resource allocation roles.
[ https://issues.apache.org/jira/browse/MESOS-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Conway updated MESOS-6375: --- Comment: was deleted (was: https://reviews.apache.org/r/57254/) > Support hierarchical resource allocation roles. > --- > > Key: MESOS-6375 > URL: https://issues.apache.org/jira/browse/MESOS-6375 > Project: Mesos > Issue Type: Epic > Components: allocation >Reporter: Benjamin Mahler >Assignee: Neil Conway > > Currently mesos provides a non-hierarchical resource allocation model, in > which all roles are siblings of one another. > Organizations often have a need for hierarchical resource allocation > constraints, whether for fair sharing of resources or for specifying quota > constraints. > Consider the following fair sharing hierarchy based on "shares": > {noformat} > ^ ^ > / \ / \ > / \ / \ >eng (3) sales (1) => eng (75%) sales (25%) > ^ ^ >/ \ / \ > / \ / \ > ads (2)build (1) ads (66%) build (33%) > {noformat} > The hierarchy specifies that the engineering organization should get 3x as > many resources as sales, and within these resources the ads team should get > 2x as many resources as the build team. The implication of this is that, if > the ads team is not using some of its resources, the build team and > engineering organization will be able to use these resources before the sales > organization can. Without a hierarchy, the resources unused by the ads team > would be re-distributed among all other roles (rather than only its siblings). > Quota can also apply in a hierarchical manner: > {noformat} > ^ > / \ > / \ >eng (90 cpus) sales (10 cpus) > ^ >/ \ > / \ > ads (50 cpus) build (10 cpus) > {noformat} > See https://people.eecs.berkeley.edu/~alig/papers/h-drf.pdf for some > discussion w.r.t. sharing resources in a hierarchical model. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7026) Update authorization / authorization-filtering to handle hierarchical roles.
[ https://issues.apache.org/jira/browse/MESOS-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier reassigned MESOS-7026: --- Assignee: Benjamin Bannier > Update authorization / authorization-filtering to handle hierarchical roles. > > > Key: MESOS-7026 > URL: https://issues.apache.org/jira/browse/MESOS-7026 > Project: Mesos > Issue Type: Task > Components: agent, HTTP API, master >Reporter: Benjamin Mahler >Assignee: Benjamin Bannier > > Authorization and endpoint filtering will need to be updated in order to > allow the authorization to be performed in a hierarchical manner (e.g. a user > can see all beneath /eng/* vs. a user can see all beneath /eng/frontend/*). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7078) Benchmarks to validate perf impact of hierarchical sorting
[ https://issues.apache.org/jira/browse/MESOS-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Conway reassigned MESOS-7078: -- Assignee: Neil Conway > Benchmarks to validate perf impact of hierarchical sorting > -- > > Key: MESOS-7078 > URL: https://issues.apache.org/jira/browse/MESOS-7078 > Project: Mesos > Issue Type: Task >Reporter: Neil Conway >Assignee: Neil Conway > Labels: mesosphere > > Depending on how deeply we need to change the sorter/allocator, we should > ensure we take the time to run the existing benchmarks (and perhaps write new > benchmarks) to ensure we don't regress performance for existing > sorter/allocator use cases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892908#comment-15892908 ] Neil Conway commented on MESOS-7197: [~bbannier] Interesting, I'll take a look. Did the issue also repro in a non-optimized build? > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Assignee: Neil Conway >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Conway reassigned MESOS-7197: -- Assignee: Neil Conway > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Assignee: Neil Conway >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7120) Add an Agent API call to cleanup nested container artifacts
[ https://issues.apache.org/jira/browse/MESOS-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-7120: - Shepherd: Alexander Rukletsov > Add an Agent API call to cleanup nested container artifacts > --- > > Key: MESOS-7120 > URL: https://issues.apache.org/jira/browse/MESOS-7120 > Project: Mesos > Issue Type: Improvement > Components: agent, containerization >Reporter: Gastón Kleiman >Assignee: Gastón Kleiman > Labels: debugging, health-check, mesosphere > > Executors and operators should be able to request the agent to cleanup all > the artifacts (i.e., the sandbox, runtime dirs, checkpointed info, etc.) > related to a nested container. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7200) Add support for hierarchical roles to the local authorizer
[ https://issues.apache.org/jira/browse/MESOS-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier updated MESOS-7200: Shepherd: Adam B > Add support for hierarchical roles to the local authorizer > -- > > Key: MESOS-7200 > URL: https://issues.apache.org/jira/browse/MESOS-7200 > Project: Mesos > Issue Type: Improvement > Components: master >Reporter: Benjamin Bannier >Assignee: Benjamin Bannier > Labels: authorization > > We should update the local authorizer so that role values for role-based > actions matching whole role subhierarchies are understood, e.g., given roles > {{a/b/c}}, {{a/b/d}} and {{a/e}} it should be possible to specify a role > value {{a/b/%}} matching actions on roles {{a/b/c}} and {{a/b/d}}, or a value > {{a/%}} matching actions on all above roles. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7004) Enable multiple HTTP authenticator modules
[ https://issues.apache.org/jira/browse/MESOS-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan updated MESOS-7004: - Sprint: Mesosphere Sprint 50, Mesosphere Sprint 51 (was: Mesosphere Sprint 50, Mesosphere Sprint 51, Mesosphere Sprint 52) > Enable multiple HTTP authenticator modules > -- > > Key: MESOS-7004 > URL: https://issues.apache.org/jira/browse/MESOS-7004 > Project: Mesos > Issue Type: Task > Components: modules, security >Reporter: Greg Mann >Assignee: Greg Mann > Labels: libprocess, module, security > > To accommodate executor authentication, we will add support for the loading > of multiple authenticator modules. The {{--http_authenticators}} flag is > already set up for this, but we must relax the constraint in Mesos which > enforces just a single authenticator, and libprocess must implement this > infrastructure. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7200) Add support for hierarchical roles to the local authorizer
Benjamin Bannier created MESOS-7200: --- Summary: Add support for hierarchical roles to the local authorizer Key: MESOS-7200 URL: https://issues.apache.org/jira/browse/MESOS-7200 Project: Mesos Issue Type: Improvement Components: master Reporter: Benjamin Bannier We should update the local authorizer so that role values for role-based actions matching whole role subhierarchies are understood, e.g., given roles {{a/b/c}}, {{a/b/d}} and {{a/e}} it should be possible to specify a role value {{a/b/%}} matching actions on roles {{a/b/c}} and {{a/b/d}}, or a value {{a/%}} matching actions on all above roles. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7200) Add support for hierarchical roles to the local authorizer
[ https://issues.apache.org/jira/browse/MESOS-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier reassigned MESOS-7200: --- Assignee: Benjamin Bannier > Add support for hierarchical roles to the local authorizer > -- > > Key: MESOS-7200 > URL: https://issues.apache.org/jira/browse/MESOS-7200 > Project: Mesos > Issue Type: Improvement > Components: master >Reporter: Benjamin Bannier >Assignee: Benjamin Bannier > Labels: authorization > > We should update the local authorizer so that role values for role-based > actions matching whole role subhierarchies are understood, e.g., given roles > {{a/b/c}}, {{a/b/d}} and {{a/e}} it should be possible to specify a role > value {{a/b/%}} matching actions on roles {{a/b/c}} and {{a/b/d}}, or a value > {{a/%}} matching actions on all above roles. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6499) Add metric to track active subscribers in operator API
[ https://issues.apache.org/jira/browse/MESOS-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892612#comment-15892612 ] Anand Mazumdar commented on MESOS-6499: --- Yes, it is. I reopened the review. > Add metric to track active subscribers in operator API > -- > > Key: MESOS-6499 > URL: https://issues.apache.org/jira/browse/MESOS-6499 > Project: Mesos > Issue Type: Improvement > Components: HTTP API >Reporter: Zhitao Li >Assignee: Zhitao Li > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6499) Add metric to track active subscribers in operator API
[ https://issues.apache.org/jira/browse/MESOS-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892603#comment-15892603 ] Zhitao Li commented on MESOS-6499: -- [~anandmazumdar], do you think this is still acceptable? I'd like to get this monitoring improvement in if possible. Review at https://reviews.apache.org/r/53267/ > Add metric to track active subscribers in operator API > -- > > Key: MESOS-6499 > URL: https://issues.apache.org/jira/browse/MESOS-6499 > Project: Mesos > Issue Type: Improvement > Components: HTTP API >Reporter: Zhitao Li >Assignee: Zhitao Li > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7003) Introduce a 'Principal' type
[ https://issues.apache.org/jira/browse/MESOS-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-7003: - Summary: Introduce a 'Principal' type (was: Introduce the a 'Principal' type) > Introduce a 'Principal' type > > > Key: MESOS-7003 > URL: https://issues.apache.org/jira/browse/MESOS-7003 > Project: Mesos > Issue Type: Task > Components: executor, security >Reporter: Greg Mann >Assignee: Greg Mann > Labels: executor, security > > We will introduce a new type to represent the identity of an authenticated > entity in Mesos: the {{Principal}}. To accomplish this, the following should > be done: > * Add the new {{Principal}} type > * Update the {{AuthenticationResult}} to use {{Principal}} > * Update all authenticated endpoint handlers to handle this new type > * Update the default authenticator modules to use the new type -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-3917) Configure and load SSL key password from configuration file
[ https://issues.apache.org/jira/browse/MESOS-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B reassigned MESOS-3917: - Assignee: (was: Disha Singh) > Configure and load SSL key password from configuration file > --- > > Key: MESOS-3917 > URL: https://issues.apache.org/jira/browse/MESOS-3917 > Project: Mesos > Issue Type: Bug > Components: agent >Affects Versions: 0.23.0 >Reporter: Timothy Chen > Labels: mesosphere > > Currently when SSL enabled and password enabled on SSL key file, launching a > task fails as the executor inherits SSL config and blocks on stdin asking for > SSL key password. > To avoid this we should allow password information to be loaded from a config > file and also pass the file information to executors and other processes that > wants SSL communication. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7003) Introduce the a 'Principal' type
[ https://issues.apache.org/jira/browse/MESOS-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-7003: - Description: We will introduce a new type to represent the identity of an authenticated entity in Mesos: the {{Principal}}. To accomplish this, the following should be done: * Add the new {{Principal}} type * Update the {{AuthenticationResult}} to use {{Principal}} * Update all authenticated endpoint handlers to handle this new type * Update the default authenticator modules to use the new type was: We will introduce a new type to represent the identity of an authenticated entity in Mesos: the {{Principal}}. To accomplish this, the following should be done: * Add the new Principal type * Update the {{AuthenticationResult}} to use {{Principal}} * Update all authenticated endpoint handlers to handle this new type * Update the default authenticator modules to use the new type > Introduce the a 'Principal' type > > > Key: MESOS-7003 > URL: https://issues.apache.org/jira/browse/MESOS-7003 > Project: Mesos > Issue Type: Task > Components: executor, security >Reporter: Greg Mann >Assignee: Greg Mann > Labels: executor, security > > We will introduce a new type to represent the identity of an authenticated > entity in Mesos: the {{Principal}}. To accomplish this, the following should > be done: > * Add the new {{Principal}} type > * Update the {{AuthenticationResult}} to use {{Principal}} > * Update all authenticated endpoint handlers to handle this new type > * Update the default authenticator modules to use the new type -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7003) Introduce the a 'Principal' type
[ https://issues.apache.org/jira/browse/MESOS-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-7003: - Summary: Introduce the a 'Principal' type (was: Introduce the AuthenticationContext) > Introduce the a 'Principal' type > > > Key: MESOS-7003 > URL: https://issues.apache.org/jira/browse/MESOS-7003 > Project: Mesos > Issue Type: Task > Components: executor, security >Reporter: Greg Mann >Assignee: Greg Mann > Labels: executor, security > > We will introduce a new type to represent the identity of an authenticated > entity in Mesos: the {{AuthenticationContext}}. To accomplish this, the > following should be done: > * Add the new AuthenticationContext type > * Update the AuthenticationResult type to use the AuthenticationContext > * Update all authenticated endpoint handlers to handle this new type > * Update the default authenticator modules to use the new type -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7003) Introduce the a 'Principal' type
[ https://issues.apache.org/jira/browse/MESOS-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann updated MESOS-7003: - Description: We will introduce a new type to represent the identity of an authenticated entity in Mesos: the {{Principal}}. To accomplish this, the following should be done: * Add the new Principal type * Update the {{AuthenticationResult}} to use {{Principal}} * Update all authenticated endpoint handlers to handle this new type * Update the default authenticator modules to use the new type was: We will introduce a new type to represent the identity of an authenticated entity in Mesos: the {{AuthenticationContext}}. To accomplish this, the following should be done: * Add the new AuthenticationContext type * Update the AuthenticationResult type to use the AuthenticationContext * Update all authenticated endpoint handlers to handle this new type * Update the default authenticator modules to use the new type > Introduce the a 'Principal' type > > > Key: MESOS-7003 > URL: https://issues.apache.org/jira/browse/MESOS-7003 > Project: Mesos > Issue Type: Task > Components: executor, security >Reporter: Greg Mann >Assignee: Greg Mann > Labels: executor, security > > We will introduce a new type to represent the identity of an authenticated > entity in Mesos: the {{Principal}}. To accomplish this, the following should > be done: > * Add the new Principal type > * Update the {{AuthenticationResult}} to use {{Principal}} > * Update all authenticated endpoint handlers to handle this new type > * Update the default authenticator modules to use the new type -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6517) Health checking only on 127.0.0.1 is limiting.
[ https://issues.apache.org/jira/browse/MESOS-6517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892523#comment-15892523 ] Deshi Xiao commented on MESOS-6517: --- what decision for it. please do it action asap. > Health checking only on 127.0.0.1 is limiting. > -- > > Key: MESOS-6517 > URL: https://issues.apache.org/jira/browse/MESOS-6517 > Project: Mesos > Issue Type: Improvement >Reporter: Alexander Rukletsov > Labels: health-check, mesosphere > > As of Mesos 1.1.0, HTTP and TCP health checks always use 127.0.0.1 as the > target IP. This is not configurable. As a result, tasks should listen on all > interfaces if they want to support HTTP and TCP health checks. However, there > might be some cases where tasks or containers will end up binding to a > specific IP address. > To make health checking more robust we can: > * look at all interfaces in a given network namespace and do health check on > all the IP addresses; > * allow users to specify the IP to health check; > * deduce the target IP from task's discovery information. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7120) Add an Agent API call to cleanup nested container artifacts
[ https://issues.apache.org/jira/browse/MESOS-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gastón Kleiman reassigned MESOS-7120: - Assignee: Gastón Kleiman > Add an Agent API call to cleanup nested container artifacts > --- > > Key: MESOS-7120 > URL: https://issues.apache.org/jira/browse/MESOS-7120 > Project: Mesos > Issue Type: Improvement > Components: agent, containerization >Reporter: Gastón Kleiman >Assignee: Gastón Kleiman > Labels: debugging, health-check, mesosphere > > Executors and operators should be able to request the agent to cleanup all > the artifacts (i.e., the sandbox, runtime dirs, checkpointed info, etc.) > related to a nested container. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7120) Add an Agent API call to cleanup nested container artifacts
[ https://issues.apache.org/jira/browse/MESOS-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gastón Kleiman updated MESOS-7120: -- Sprint: Mesosphere Sprint 52 > Add an Agent API call to cleanup nested container artifacts > --- > > Key: MESOS-7120 > URL: https://issues.apache.org/jira/browse/MESOS-7120 > Project: Mesos > Issue Type: Improvement > Components: agent, containerization >Reporter: Gastón Kleiman > Labels: debugging, health-check, mesosphere > > Executors and operators should be able to request the agent to cleanup all > the artifacts (i.e., the sandbox, runtime dirs, checkpointed info, etc.) > related to a nested container. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7199) Allow Docker private registry credentials to be passed from framework with Docker containerizer
A. Dukhovniy created MESOS-7199: --- Summary: Allow Docker private registry credentials to be passed from framework with Docker containerizer Key: MESOS-7199 URL: https://issues.apache.org/jira/browse/MESOS-7199 Project: Mesos Issue Type: Improvement Components: containerization, docker Reporter: A. Dukhovniy Priority: Minor Currently it is planned to allow docker private registry credentials to be passed from framework but only when using mesos containerizer: [MESOS-4242|https://issues.apache.org/jira/browse/MESOS-4242], [MESOS-7088|https://issues.apache.org/jira/browse/MESOS-7088]. This should be implemented for docker containerizer too to provide a consistent way of accessing private registries independent of the containerizer. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7198) After the mesos-slave is restarted, the mesos-slave can not get the container's environment variable.
wharton liu created MESOS-7198: -- Summary: After the mesos-slave is restarted, the mesos-slave can not get the container's environment variable. Key: MESOS-7198 URL: https://issues.apache.org/jira/browse/MESOS-7198 Project: Mesos Issue Type: Bug Components: agent Affects Versions: 0.24.1 Reporter: wharton liu The hook is called by the environment variable of the container, the hook will be called after the container exits. However, after the mesos-slave is restarted, the mesos-slave can not get the container's environment variable. mesos 0.24.1 file: src/slave/containerizer/docker.cpp fuction: Future DockerContainerizerProcess::_recover() Code: Container* container = new Container(containerId); -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891917#comment-15891917 ] Benjamin Bannier edited comment on MESOS-7197 at 3/2/17 9:40 AM: - [~bmerry]: I was able to reproduce this; in an optimized build created with clang-trunk I see a critical check failure even for {{cpus:0.0001;mem:1}}. [~neilc]: These sub-permille numbers appear to be below the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? was (Author: bbannier): [~bmerry]: I was able to reproduce this; in an optimized build created with clang-trunk I see a critical check failure even for {{cpus:0.0001;mem:1}}. [~neilc]: These permille numbers appear to be below the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891917#comment-15891917 ] Benjamin Bannier edited comment on MESOS-7197 at 3/2/17 9:35 AM: - [~bmerry]: I was able to reproduce this; in an optimized build created with clang-trunk I see a critical check failure even for {{cpus:0.0001;mem:1}}. [~neilc]: These permille numbers appear to be below the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? was (Author: bbannier): [~bmerry]: I was able to reproduce this; in an optimized build created with clang-trunk I see a critical check failure even for {{cpus:0.001;mem:1}}. [~neilc]: These permille numbers appear suspiciously close to the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891917#comment-15891917 ] Benjamin Bannier edited comment on MESOS-7197 at 3/2/17 9:33 AM: - [~bmerry]: I was able to reproduce this; in an optimized build created with clang-trunk I see a critical check failure even for {{cpus:0.001;mem:1}}. [~neilc]: These permille numbers appear suspiciously close to the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? was (Author: bbannier): [~bmerry]: I was able to reproduce this, and with and an optimized build created with clang-trunk shows a critical check failure even for {{cpus:0.001;mem:1}}. [~neilc]: These permille numbers appear suspiciously close to the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891917#comment-15891917 ] Benjamin Bannier commented on MESOS-7197: - [~bmerry]: I was able to reproduce this, and with and an optimized build created with clang-trunk shows a critical check failure even for {{cpus:0.001;mem:1}}. [~neilc]: These permille numbers appear suspiciously close to the edge of what we ignore for fixed point resource math. Is this a bug in the fixed point math or are we just missing validation? > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Bannier updated MESOS-7197: Affects Version/s: 1.2.0 > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0, 1.2.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891890#comment-15891890 ] Benjamin Bannier commented on MESOS-7197: - This looks related to MESOS-4687. > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7197) Requesting tiny amount of CPU crashes master
[ https://issues.apache.org/jira/browse/MESOS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891856#comment-15891856 ] Bruce Merry commented on MESOS-7197: By the way, I filed this as "critical" since the descriptions for the severity indicated that was appropriate for a crash, but this isn't actually critical for us because it's easy to work around (the only reason we were requesting such a tiny CPU amount is that it's calculated). > Requesting tiny amount of CPU crashes master > > > Key: MESOS-7197 > URL: https://issues.apache.org/jira/browse/MESOS-7197 > Project: Mesos > Issue Type: Bug > Components: allocation >Affects Versions: 1.1.0 > Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos >Reporter: Bruce Merry >Priority: Critical > > If a task is submitted with a tiny CPU request e.g. 0.0004, then when it > completes the master crashes due to a CHECK failure: > {noformat} > F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: > allocations[name].resources[slaveId].contains(resources) > {noformat} > I can reproduce this with the following command: > {noformat} > mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest > --resources='cpus:0.0004;mem:128' > {noformat} > If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7197) Requesting tiny amount of CPU crashes master
Bruce Merry created MESOS-7197: -- Summary: Requesting tiny amount of CPU crashes master Key: MESOS-7197 URL: https://issues.apache.org/jira/browse/MESOS-7197 Project: Mesos Issue Type: Bug Components: allocation Affects Versions: 1.1.0 Environment: Ubuntu 14.04, using Mesosphere PPA to install Mesos Reporter: Bruce Merry Priority: Critical If a task is submitted with a tiny CPU request e.g. 0.0004, then when it completes the master crashes due to a CHECK failure: {noformat} F0302 10:48:26.654909 15391 sorter.cpp:291] Check failed: allocations[name].resources[slaveId].contains(resources) {noformat} I can reproduce this with the following command: {noformat} mesos-execute --command='sleep 5' --master=$MASTER --name=crashtest --resources='cpus:0.0004;mem:128' {noformat} If I replace 0.0004 with 0.001 the issue no longer occurs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)