[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539391#comment-16539391
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user thzois commented on the issue:

https://github.com/apache/mesos/pull/301
  
Hello, I think that everything is OK now. Can you confirm this? I will also 
create a new pull request for the "REVIVE" call.


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (MESOS-9068) Add a metrics benchmark in libprocess.

2018-07-10 Thread Benjamin Mahler (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Mahler reassigned MESOS-9068:
--

Assignee: Benjamin Mahler

> Add a metrics benchmark in libprocess.
> --
>
> Key: MESOS-9068
> URL: https://issues.apache.org/jira/browse/MESOS-9068
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess
>Reporter: Benjamin Mahler
>Assignee: Benjamin Mahler
>Priority: Major
>  Labels: performance
>
> Libprocess metrics scalability is being worked on and currently a mesos 
> specific benchmark is being used. It would be nice to have a benchmark in 
> libprocess to iterate on the library improvements in a more minimal benchmark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (MESOS-9068) Add a metrics benchmark in libprocess.

2018-07-10 Thread Benjamin Mahler (JIRA)
Benjamin Mahler created MESOS-9068:
--

 Summary: Add a metrics benchmark in libprocess.
 Key: MESOS-9068
 URL: https://issues.apache.org/jira/browse/MESOS-9068
 Project: Mesos
  Issue Type: Task
  Components: libprocess
Reporter: Benjamin Mahler


Libprocess metrics scalability is being worked on and currently a mesos 
specific benchmark is being used. It would be nice to have a benchmark in 
libprocess to iterate on the library improvements in a more minimal benchmark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (MESOS-9067) Improve performance of json parsing by avoiding conversion cost.

2018-07-10 Thread Benjamin Mahler (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539159#comment-16539159
 ] 

Benjamin Mahler edited comment on MESOS-9067 at 7/10/18 7:43 PM:
-

{noformat}
commit 7918442b6c15ca31a401247850c5ad2d10d12317
Author: Benjamin Mahler 
Date:   Mon Jul 9 18:21:03 2018 -0700

Updated json parsing to avoid copying between picojson and stout JSON.

When looking at some profiling data, it looks as though we approximately
double the cost of JSON parsing due to having to convert from
`picojson::value` to `JSON::Value`.

Michael Park pointed me to the parsing "context" that's customizable in
picojson. This patch replaces our conversion with a parsing context in
order to parse directly into JSON::Value and avoid copying.

Review: https://reviews.apache.org/r/67861
{noformat}


was (Author: bmahler):
{noformat}
commit 7918442b6c15ca31a401247850c5ad2d10d12317
Author: Benjamin Mahler 
Date:   Mon Jul 9 18:21:03 2018 -0700

Updated json parsing to avoid copying between picojson and stout JSON.

When looking at some profiling data, it looks as though we approximately
double the cost of JSON parsing due to having to convert from
`picojson::value` to `JSON::Value`.

Michael Park pointed me to the parsing "context" that's customizable in
picojson. This patch replaces our conversion with a parsing context in
order to parse directly into JSON::Value and avoid copying.

Review: https://reviews.apache.org/r/67861
{noforamt}

> Improve performance of json parsing by avoiding conversion cost.
> 
>
> Key: MESOS-9067
> URL: https://issues.apache.org/jira/browse/MESOS-9067
> Project: Mesos
>  Issue Type: Improvement
>  Components: stout
>Reporter: Benjamin Mahler
>Assignee: Benjamin Mahler
>Priority: Major
>  Labels: performance
> Fix For: 1.7.0
>
>
> Stout's JSON parsing function parses into picojson::value followed by 
> converting to JSON::Value. This carries a significant conversion cost (almost 
> doubling the parse time).
> We can leverage picojson's parsing context to avoid this cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (MESOS-9067) Improve performance of json parsing by avoiding conversion cost.

2018-07-10 Thread Benjamin Mahler (JIRA)
Benjamin Mahler created MESOS-9067:
--

 Summary: Improve performance of json parsing by avoiding 
conversion cost.
 Key: MESOS-9067
 URL: https://issues.apache.org/jira/browse/MESOS-9067
 Project: Mesos
  Issue Type: Improvement
  Components: stout
Reporter: Benjamin Mahler


Stout's JSON parsing function parses into picojson::value followed by 
converting to JSON::Value. This carries a significant conversion cost (almost 
doubling the parse time).

We can leverage picojson's parsing context to avoid this cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (MESOS-9067) Improve performance of json parsing by avoiding conversion cost.

2018-07-10 Thread Benjamin Mahler (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Mahler reassigned MESOS-9067:
--

Assignee: Benjamin Mahler

> Improve performance of json parsing by avoiding conversion cost.
> 
>
> Key: MESOS-9067
> URL: https://issues.apache.org/jira/browse/MESOS-9067
> Project: Mesos
>  Issue Type: Improvement
>  Components: stout
>Reporter: Benjamin Mahler
>Assignee: Benjamin Mahler
>Priority: Major
>  Labels: performance
>
> Stout's JSON parsing function parses into picojson::value followed by 
> converting to JSON::Value. This carries a significant conversion cost (almost 
> doubling the parse time).
> We can leverage picojson's parsing context to avoid this cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (MESOS-7397) apply-reviews.py silently fails when using chain mode.

2018-07-10 Thread Till Toenshoff (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Till Toenshoff reassigned MESOS-7397:
-

Assignee: Till Toenshoff

> apply-reviews.py silently fails when using chain mode.
> --
>
> Key: MESOS-7397
> URL: https://issues.apache.org/jira/browse/MESOS-7397
> Project: Mesos
>  Issue Type: Bug
>Reporter: Till Toenshoff
>Assignee: Till Toenshoff
>Priority: Minor
> Fix For: 1.7.0
>
>
> When using {{apply-reviews.py}} in chain mode {{--chain}} it suppresses error 
> messages when patches do not apply.
> Example on {{1.0.x}}:
> {noformat}
> $ ./support/apply-reviews.py -c -r 57764
> $
> {noformat} 
> If used without chain mode, the error message is properly displayed:
> {noformat}
> $ ./support/apply-reviews.py -r 57764
> 2017-04-18 11:45:58 URL:https://reviews.apache.org/r/57764/diff/raw/ 
> [1067/1067] -> "57764.patch" [1]
> error: patch failed: src/launcher/posix/executor.cpp:116
> error: src/launcher/posix/executor.cpp: patch does not apply
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7397) apply-reviews.py silently fails when using chain mode.

2018-07-10 Thread Till Toenshoff (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539006#comment-16539006
 ] 

Till Toenshoff commented on MESOS-7397:
---

{noformat}
commit 997e280b57232b5f9b12e0e9dfd0abfc9d6c1085 (HEAD -> master)
Author: Till Toenshoff toensh...@me.com
Date:   Tue Jul 10 19:33:58 2018 +0200

Updated apply-reviews.py messaging for submitted reviews.

Review: https://reviews.apache.org/r/67859/
{noformat}

> apply-reviews.py silently fails when using chain mode.
> --
>
> Key: MESOS-7397
> URL: https://issues.apache.org/jira/browse/MESOS-7397
> Project: Mesos
>  Issue Type: Bug
>Reporter: Till Toenshoff
>Priority: Minor
> Fix For: 1.7.0
>
>
> When using {{apply-reviews.py}} in chain mode {{--chain}} it suppresses error 
> messages when patches do not apply.
> Example on {{1.0.x}}:
> {noformat}
> $ ./support/apply-reviews.py -c -r 57764
> $
> {noformat} 
> If used without chain mode, the error message is properly displayed:
> {noformat}
> $ ./support/apply-reviews.py -r 57764
> 2017-04-18 11:45:58 URL:https://reviews.apache.org/r/57764/diff/raw/ 
> [1067/1067] -> "57764.patch" [1]
> error: patch failed: src/launcher/posix/executor.cpp:116
> error: src/launcher/posix/executor.cpp: patch does not apply
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538890#comment-16538890
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user vinodkone commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201406612
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
+Sent by the scheduler when it wants to inform Mesos master to stop sending 
offers to the framework. When Mesos receives this request it will stop sending 
offers to framework but the tasks will continue running for 
`FrameworkInfo.failover_timeout`. Once the framework has work to do, it can 
send again a `REVIVE` request.
--- End diff --

How about:

Sent by the scheduler when it doesn't need offers for a given set of its 
roles. When Mesos master receives this request, it will stop sending offers for 
the given set of roles to the framework. As a special case, if `roles` is not 
specified, all subscribed roles of this framework are suppressed.

Note that master continues to send offers to other subscribed roles of this 
framework that are not suppressed. Also, status updates about tasks, executors 
and agents are not affected by this call. 

If the scheduler wishes to receive offers for the suppressed roles again 
(e.g., it needs to schedule new workloads), it can send `REVIVE` call.


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538892#comment-16538892
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user vinodkone commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201407700
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
+Sent by the scheduler when it wants to inform Mesos master to stop sending 
offers to the framework. When Mesos receives this request it will stop sending 
offers to framework but the tasks will continue running for 
`FrameworkInfo.failover_timeout`. Once the framework has work to do, it can 
send again a `REVIVE` request.
+
+```
+SUPPRESS Request (JSON):
+POST /api/v1/scheduler  HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
+
+{
+  "framework_id": {"value" : "12220-3440-12532-2345"},
+  "type": "SUPPRESS"
+}
+
+SUPPRESS Response:
+HTTP/1.1 202 Accepted
+```
+
--- End diff --

While you are at it, can you also update the documentation about "REVIVE" 
call below. I think it's a bit outdated now. Preferably in a different PR.


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538889#comment-16538889
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user vinodkone commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201401610
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
--- End diff --

Can you move this to #500 below #REQUEST section to preserve the call enum 
order?


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538891#comment-16538891
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user vinodkone commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201406742
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
+Sent by the scheduler when it wants to inform Mesos master to stop sending 
offers to the framework. When Mesos receives this request it will stop sending 
offers to framework but the tasks will continue running for 
`FrameworkInfo.failover_timeout`. Once the framework has work to do, it can 
send again a `REVIVE` request.
+
+```
+SUPPRESS Request (JSON):
+POST /api/v1/scheduler  HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
+
+{
+  "framework_id": {"value" : "12220-3440-12532-2345"},
+  "type": "SUPPRESS"
+}
--- End diff --

Can you also include `roles` field here for completeness?


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538868#comment-16538868
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user nfnt commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201402242
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
+Sent by the scheduler when it wants to inform Mesos master to stop sending 
offers to the framework. When Mesos receives this request it will stop sending 
offers to framework but the tasks will continue running for 
`FrameworkInfo.failover_timeout`. Once the framework has work to do, it can 
send again a `REVIVE` request.
+
+```
+SUPPRESS Request (JSON):
+POST /api/v1/scheduler  HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
+
+{
+  "framework_id": {"value" : "12220-3440-12532-2345"},
+  "type": "SUPPRESS"
--- End diff --

Please re-indent. With spaces, even though some other calls in this 
document use tabs.


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538869#comment-16538869
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

Github user nfnt commented on a diff in the pull request:

https://github.com/apache/mesos/pull/301#discussion_r201402758
  
--- Diff: docs/scheduler-http-api.md ---
@@ -128,6 +128,26 @@ TEARDOWN Response:
 HTTP/1.1 202 Accepted
 ```
 
+### SUPPRESS
+Sent by the scheduler when it wants to inform Mesos master to stop sending 
offers to the framework. When Mesos receives this request it will stop sending 
offers to framework but the tasks will continue running for 
`FrameworkInfo.failover_timeout`. Once the framework has work to do, it can 
send again a `REVIVE` request.
+
+```
+SUPPRESS Request (JSON):
+POST /api/v1/scheduler  HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
+
+{
+  "framework_id": {"value" : "12220-3440-12532-2345"},
+  "type": "SUPPRESS"
+}
--- End diff --

The `Supress` call has a `roles` option. Please use an example call that 
sets a role to suppress offers for.


> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call

2018-07-10 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538584#comment-16538584
 ] 

ASF GitHub Bot commented on MESOS-7211:
---

GitHub user thzois opened a pull request:

https://github.com/apache/mesos/pull/301

Document SUPPRESS HTTP call [MESOS-7211]



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thzois/mesos master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mesos/pull/301.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #301


commit 2c9bc7270956c17b055f23b24a24a006748b7725
Author: zois 
Date:   2018-07-10T13:24:37Z

Document SUPPRESS HTTP call [MESOS-7211]




> Document SUPPRESS HTTP call
> ---
>
> Key: MESOS-7211
> URL: https://issues.apache.org/jira/browse/MESOS-7211
> Project: Mesos
>  Issue Type: Documentation
>  Components: documentation
>Affects Versions: 1.1.0
>Reporter: Bruce Merry
>Priority: Minor
>  Labels: mesosphere, newbie
>
> The documentation at 
> http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list 
> the SUPPRESS call as one of the call types, but it does seem to be 
> implemented.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (MESOS-8418) mesos-agent high cpu usage because of numerous /proc/mounts reads

2018-07-10 Thread Stephan Erb (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538309#comment-16538309
 ] 

Stephan Erb commented on MESOS-8418:


As a workaround bumping the following options seems to help reduce the 
{{mesos-agent}} load:

* container_disk_watch_interval
* disk_watch_interval
* oversubscribed_resources_interval
* qos_correction_interval_min

> mesos-agent high cpu usage because of numerous /proc/mounts reads
> -
>
> Key: MESOS-8418
> URL: https://issues.apache.org/jira/browse/MESOS-8418
> Project: Mesos
>  Issue Type: Improvement
>  Components: agent, containerization
>Reporter: Stéphane Cottin
>Priority: Major
>  Labels: containerizer, performance
>
> /proc/mounts is read many, many times from 
> src/(linux/fs|linux/cgroups|slave/slave).cpp.
> When using overlayfs, the /proc/mounts contents can become quite large. 
> As an example, one of our Q/A single node running ~150 tasks,  have a 361 
> lines/ 201299 chars  /proc/mounts file.
> This 200kB file is read on this node about 25 to 150 times per second. This 
> is a (huge) waste of cpu and I/O time.
> Most of these calls are related to cgroups.
> Please consider these proposals :
> 1/ Is /proc/mounts mandatory for cgroups ? 
> We already have cgroup subsystems list from /proc/cgroups.
> The only compelling information from /proc/mounts seems to be the root mount 
> point, 
> /sys/fs/cgroup/, which could be obtained by a unique read on agent start.
> 2/ use /proc/self/mountstats
> {noformat}
> wc /proc/self/mounts /proc/self/mountstats
> 361 2166 201299 /proc/self/mounts
> 361 2888 50200 /proc/self/mountstats
> {noformat}
> {noformat}
> grep cgroup /proc/self/mounts
> cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
> cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
> cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
> cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
> cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
> cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
> cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
> cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
> cgroup /sys/fs/cgroup/net_cls cgroup rw,relatime,net_cls 0 0
> cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event 0 0
> cgroup /sys/fs/cgroup/net_prio cgroup rw,relatime,net_prio 0 0
> cgroup /sys/fs/cgroup/pids cgroup rw,relatime,pids 0 0
> {noformat}
> {noformat}
> grep cgroup /proc/self/mountstats
> device cgroup mounted on /sys/fs/cgroup with fstype tmpfs
> device cgroup mounted on /sys/fs/cgroup/cpuset with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/cpu with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/cpuacct with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/blkio with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/memory with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/devices with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/freezer with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/net_cls with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/perf_event with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/net_prio with fstype cgroup
> device cgroup mounted on /sys/fs/cgroup/pids with fstype cgroup
> {noformat}
> This file contains all the required information, and is 4x smaller
> 3/ microcaching
> Caching cgroups data for just 1 second would be a huge perfomance 
> improvement, but i'm not aware of the possible side effects.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (MESOS-7947) Add GC capability to nested containers

2018-07-10 Thread Qian Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qian Zhang reassigned MESOS-7947:
-

Assignee: Qian Zhang

> Add GC capability to nested containers
> --
>
> Key: MESOS-7947
> URL: https://issues.apache.org/jira/browse/MESOS-7947
> Project: Mesos
>  Issue Type: Improvement
>  Components: executor
>Reporter: Chun-Hung Hsiao
>Assignee: Qian Zhang
>Priority: Major
>
> We should extend the existing API or add a new API for nested containers for 
> an executor to tell the Mesos agent that a nested container is no longer 
> needed and can be scheduled for GC.
> Related issue: MESOS-7939



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)