[mesos] branch master updated: Updated `release-guide.md`.
This is an automated email from the ASF dual-hosted git repository. qianzhang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git The following commit(s) were added to refs/heads/master by this push: new 57f9b46 Updated `release-guide.md`. 57f9b46 is described below commit 57f9b46a743a1916f0bd56c8c91e881c9ad9b7d3 Author: Qian Zhang AuthorDate: Fri Aug 30 17:32:31 2019 +0800 Updated `release-guide.md`. Removed unnecessary steps and added the steps for pushing changes to remote master and release branch. Review: https://reviews.apache.org/r/71415 --- docs/release-guide.md | 22 -- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/release-guide.md b/docs/release-guide.md index bae7414..84e9f85 100644 --- a/docs/release-guide.md +++ b/docs/release-guide.md @@ -62,7 +62,7 @@ This guide describes the process of doing an official release of Mesos. ## Preparation 1. If this is a regular release, create a new release branch (..x) - based off the current master + based off the current master. $ git checkout origin/master -b X.Y.x @@ -132,27 +132,29 @@ This guide describes the process of doing an official release of Mesos. the release. Do not forget to remove "(WIP)" suffix from the release notes' title. -5. Run `make && support/generate-endpoint-help.py` and commit any resulting - changes. - -6. Update and commit `docs/configuration.md` to reflect the current state of +5. Update and commit `docs/configuration.md` to reflect the current state of the master, agent, and configure flags. Update it on master branch and then cherry pick onto the release branch. -7. If this is a regular release, update and commit `docs/upgrades.md` with +6. If this is a regular release, update and commit `docs/upgrades.md` with instructions about how to upgrade a live cluster from the previous release - version to this release version. + version to this release version. Update it on master branch and then cherry + pick onto the release branch. -8. If this is a regular release, please ensure that user documentation has been +7. If this is a regular release, please ensure that user documentation has been added for any new features. -9. Make sure that for any updates of the API, specifically the scheduler API, +8. Make sure that for any updates of the API, specifically the scheduler API, the public mesos protobuf definitions are part of both, `include/mesos` as well as `include/mesos/v1`. **NOTE:** This might actually demand code updates if any omissions were identified. +9. Push your changes on master branch and the new release branch if this is a + regular release, push your changes on the existing release branch if this is + a patch release. + ## Tagging and Voting the Release Candidate @@ -208,7 +210,7 @@ This guide describes the process of doing an official release of Mesos. $ git checkout X.Y.x $ git cherry-pick abcdefgh... -4. Now go back up to the "Tagging the Release Candidate" section and repeat. +4. Now go back up to the "Tagging and Voting the Release Candidate" section and repeat. ## Releasing the Release Candidate
[mesos-site] branch asf-site updated: Updated the website built from mesos SHA: 9297e2d3b.
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/mesos-site.git The following commit(s) were added to refs/heads/asf-site by this push: new b54108e Updated the website built from mesos SHA: 9297e2d3b. b54108e is described below commit b54108e32259547a955bfee0157e93edad242645 Author: jenkins AuthorDate: Fri Aug 30 15:51:55 2019 + Updated the website built from mesos SHA: 9297e2d3b. --- content/documentation/latest/release-guide/index.html | 4 ++-- content/documentation/release-guide/index.html| 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/documentation/latest/release-guide/index.html b/content/documentation/latest/release-guide/index.html index aff95be..5439831 100644 --- a/content/documentation/latest/release-guide/index.html +++ b/content/documentation/latest/release-guide/index.html @@ -274,7 +274,7 @@ regressions: NOTE: X.Y.Z is based on http://semver.org/;>semantic versioning scheme. R is release candidate version that starts with 1. Tag the release externally and deploy the corresponding JAR to the -https://repository.apache.org;>Apache maven repository. It is recommended +https://repository.apache.org;>Apache Maven repository. It is recommended to use the support/vote.sh script to accomplish this. $ ./support/vote.sh X.Y.Z R @@ -300,7 +300,7 @@ we allow a longer vote, to allow more time for integration testing. Send a reply to the original VOTE email with subject [RESULT][VOTE] Release Apache Mesos X.Y.Z (rcN) and mention that the vote is canceled. -Go to https://repository.apache.org/#stagingRepositories and Drop the staging repository containing the JAR (you can find the link to the staging repository in the original VOTE email). +Go to https://repository.apache.org/#stagingRepositories;>Apache Maven staging repositories and Drop the staging repository containing the JAR (you can find the exact link to the staging repository in the original VOTE email). Track the issues as new JIRAs for the release. diff --git a/content/documentation/release-guide/index.html b/content/documentation/release-guide/index.html index 4281b35..1c145f8 100644 --- a/content/documentation/release-guide/index.html +++ b/content/documentation/release-guide/index.html @@ -274,7 +274,7 @@ regressions: NOTE: X.Y.Z is based on http://semver.org/;>semantic versioning scheme. R is release candidate version that starts with 1. Tag the release externally and deploy the corresponding JAR to the -https://repository.apache.org;>Apache maven repository. It is recommended +https://repository.apache.org;>Apache Maven repository. It is recommended to use the support/vote.sh script to accomplish this. $ ./support/vote.sh X.Y.Z R @@ -300,7 +300,7 @@ we allow a longer vote, to allow more time for integration testing. Send a reply to the original VOTE email with subject [RESULT][VOTE] Release Apache Mesos X.Y.Z (rcN) and mention that the vote is canceled. -Go to https://repository.apache.org/#stagingRepositories and Drop the staging repository containing the JAR (you can find the link to the staging repository in the original VOTE email). +Go to https://repository.apache.org/#stagingRepositories;>Apache Maven staging repositories and Drop the staging repository containing the JAR (you can find the exact link to the staging repository in the original VOTE email). Track the issues as new JIRAs for the release.
[mesos-site] branch asf-site updated: Updated the website built from mesos SHA: e5bf1b61e.
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/mesos-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 4cd1597 Updated the website built from mesos SHA: e5bf1b61e. 4cd1597 is described below commit 4cd1597ca633308b49894d3bd95372277d17e4a1 Author: jenkins AuthorDate: Fri Aug 30 15:23:14 2019 + Updated the website built from mesos SHA: e5bf1b61e. --- content/documentation/latest/upgrades/index.html | 8 ++-- content/documentation/upgrades/index.html| 8 ++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/content/documentation/latest/upgrades/index.html b/content/documentation/latest/upgrades/index.html index 58f3395..4d311a7 100644 --- a/content/documentation/latest/upgrades/index.html +++ b/content/documentation/latest/upgrades/index.html @@ -634,8 +634,12 @@ See docs/maintenance - -* The Mesos agent now requires the new AGENT_DRAINING feature. This capability is set by default, but if the --agent_features flag is specified explicitly, AGENT_DRAINING must be included. + + + +The Mesos agent now requires the new AGENT_DRAINING feature. This capability is set by default, but if the --agent_features flag is specified explicitly, AGENT_DRAINING must be included. + + diff --git a/content/documentation/upgrades/index.html b/content/documentation/upgrades/index.html index cc227e8..934c8cd 100644 --- a/content/documentation/upgrades/index.html +++ b/content/documentation/upgrades/index.html @@ -634,8 +634,12 @@ See docs/maintenance - -* The Mesos agent now requires the new AGENT_DRAINING feature. This capability is set by default, but if the --agent_features flag is specified explicitly, AGENT_DRAINING must be included. + + + +The Mesos agent now requires the new AGENT_DRAINING feature. This capability is set by default, but if the --agent_features flag is specified explicitly, AGENT_DRAINING must be included. + +
[mesos] branch master updated: Fixed the link to RAO staging repo in the release guide.
This is an automated email from the ASF dual-hosted git repository. vinodkone pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git The following commit(s) were added to refs/heads/master by this push: new 9297e2d Fixed the link to RAO staging repo in the release guide. 9297e2d is described below commit 9297e2d3b0d44b553fc89bcf5f6109c76cc53668 Author: Vinod Kone AuthorDate: Fri Aug 30 10:10:05 2019 -0500 Fixed the link to RAO staging repo in the release guide. --- docs/release-guide.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/release-guide.md b/docs/release-guide.md index ce9c7c1..bae7414 100644 --- a/docs/release-guide.md +++ b/docs/release-guide.md @@ -173,7 +173,7 @@ This guide describes the process of doing an official release of Mesos. scheme. `R` is release candidate version that starts with 1. 4. Tag the release externally and deploy the corresponding JAR to the - [Apache maven repository](https://repository.apache.org). It is recommended + [Apache Maven repository](https://repository.apache.org). It is recommended to use the `support/vote.sh` script to accomplish this. $ ./support/vote.sh X.Y.Z R @@ -195,7 +195,7 @@ This guide describes the process of doing an official release of Mesos. 1. If the vote does not pass (any -1s or showstopper bugs) 1. Send a reply to the original VOTE email with subject "[RESULT][VOTE] Release Apache Mesos X.Y.Z (rcN)" and mention that the vote is canceled. - 2. Go to https://repository.apache.org/#stagingRepositories and "Drop" the staging repository containing the JAR (you can find the link to the staging repository in the original VOTE email). + 2. Go to [Apache Maven staging repositories](https://repository.apache.org/#stagingRepositories) and "Drop" the staging repository containing the JAR (you can find the exact link to the staging repository in the original VOTE email). 3. Track the issues as new JIRAs for the release. 2. When all known issues are resolved, update the `CHANGELOG` with the newly
[mesos] 03/03: Fixed formatting in the upgrade docs.
This is an automated email from the ASF dual-hosted git repository. grag pushed a commit to branch 1.9.x in repository https://gitbox.apache.org/repos/asf/mesos.git commit 5e79a584e6ec3e9e2f96e8bf418411df9dafac2e Author: Greg Mann AuthorDate: Fri Aug 30 11:48:18 2019 -0300 Fixed formatting in the upgrade docs. --- docs/upgrades.md | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/upgrades.md b/docs/upgrades.md index d36a9a4..d745752 100644 --- a/docs/upgrades.md +++ b/docs/upgrades.md @@ -522,21 +522,22 @@ We categorize the changes as follows: - * A new `DRAINING` state has been added to Mesos agents. Once an agent is draining, all tasks running on that agent are gracefully -killed and no offers for that agent are sent to schedulers, preventing the launching of new tasks. -Operators can put an agent into `DRAINING` state by using the `DRAIN_AGENT` operator API call. -See [`docs/maintenance`](maintenance.md) for details. +* A new `DRAINING` state has been added to Mesos agents. Once an agent is draining, all tasks running on that agent are gracefully + killed and no offers for that agent are sent to schedulers, preventing the launching of new tasks. + Operators can put an agent into `DRAINING` state by using the `DRAIN_AGENT` operator API call. + See [`docs/maintenance`](maintenance.md) for details. + * The Mesos agent now requires the new `AGENT_DRAINING` feature. This capability is set by default, but if the `--agent_features` flag is specified explicitly, `AGENT_DRAINING` must be included. - * A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The isolator supports setting of the `no_new_privs` bit in the container, preventing tasks from acquiring additional privileges. +* A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The isolator supports setting of the `no_new_privs` bit in the container, preventing tasks from acquiring additional privileges. - * A new [`--docker_ignore_runtime`](configuration/agent.md#docker_ignore_runtime) flag has been added. This causes the agent to ignore any runtime configuration present in Docker images. +* A new [`--docker_ignore_runtime`](configuration/agent.md#docker_ignore_runtime) flag has been added. This causes the agent to ignore any runtime configuration present in Docker images.
[mesos] branch 1.9.x updated (091f193 -> 5e79a58)
This is an automated email from the ASF dual-hosted git repository. grag pushed a change to branch 1.9.x in repository https://gitbox.apache.org/repos/asf/mesos.git. from 091f193 Updated docs for the AGENT_DRAINING capability. new 9aa7dab Removed experimental warning from UPDATE_QUOTA call. new 499a571 Updated upgrades.md to note quota limits changes. new 5e79a58 Fixed formatting in the upgrade docs. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/upgrades.md | 34 -- include/mesos/master/master.proto| 4 include/mesos/v1/master/master.proto | 4 3 files changed, 28 insertions(+), 14 deletions(-)
[mesos] 01/03: Removed experimental warning from UPDATE_QUOTA call.
This is an automated email from the ASF dual-hosted git repository.
grag pushed a commit to branch 1.9.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 9aa7dab8061292d22ac870a2cf8856dca50e31d2
Author: Benjamin Mahler
AuthorDate: Thu Aug 29 12:59:01 2019 -0400
Removed experimental warning from UPDATE_QUOTA call.
This is now a fully functional feature and is released in 1.9.
Review: https://reviews.apache.org/r/71411
---
include/mesos/master/master.proto| 4
include/mesos/v1/master/master.proto | 4
2 files changed, 8 deletions(-)
diff --git a/include/mesos/master/master.proto
b/include/mesos/master/master.proto
index 8386bd6..315809c 100644
--- a/include/mesos/master/master.proto
+++ b/include/mesos/master/master.proto
@@ -274,10 +274,6 @@ message Call {
required SlaveID slave_id = 1;
}
- // EXPERIMENTAL DO NOT USE.
- //
- // This feature is not implementation complete.
- //
// Updates quota given the provided quota configurations, these
configurations
// are applied in an all-or-nothing manner.
message UpdateQuota {
diff --git a/include/mesos/v1/master/master.proto
b/include/mesos/v1/master/master.proto
index 893162d..5c99112 100644
--- a/include/mesos/v1/master/master.proto
+++ b/include/mesos/v1/master/master.proto
@@ -275,10 +275,6 @@ message Call {
required AgentID agent_id = 1;
}
- // EXPERIMENTAL DO NOT USE.
- //
- // This feature is not implementation complete.
- //
// Updates quota given the provided quota configurations, these
configurations
// are applied in an all-or-nothing manner.
message UpdateQuota {
[mesos] 02/03: Updated upgrades.md to note quota limits changes.
This is an automated email from the ASF dual-hosted git repository. grag pushed a commit to branch 1.9.x in repository https://gitbox.apache.org/repos/asf/mesos.git commit 499a57188518e6d541f95e0d10ac264e6f83a735 Author: Benjamin Mahler AuthorDate: Thu Aug 29 12:59:54 2019 -0400 Updated upgrades.md to note quota limits changes. In particular: * UPDATE_QUOTA replaces the old SET_QUOTA and REMOVE_QUOTA calls. * Quota guarantees are still functional, but deprecated in preparation for optimistic offers. Review: https://reviews.apache.org/r/71412 --- docs/upgrades.md | 21 + 1 file changed, 21 insertions(+) diff --git a/docs/upgrades.md b/docs/upgrades.md index 0345e22..d36a9a4 100644 --- a/docs/upgrades.md +++ b/docs/upgrades.md @@ -48,6 +48,7 @@ We categorize the changes as follows: + A Quota Limits A Linux NNP isolator A hostname_validation_scheme C TLS certificate verification behaviour @@ -78,6 +79,10 @@ We categorize the changes as follows: + D SET_QUOTA and REMOVE QUOTA deprecated +in favor of UPDATE_QUOTA + D Quota guarantees deprecated in favor +of using quota limits @@ -516,6 +521,7 @@ We categorize the changes as follows: ## Upgrading from 1.8.x to 1.9.x ## + * A new `DRAINING` state has been added to Mesos agents. Once an agent is draining, all tasks running on that agent are gracefully killed and no offers for that agent are sent to schedulers, preventing the launching of new tasks. Operators can put an agent into `DRAINING` state by using the `DRAIN_AGENT` operator API call. @@ -525,17 +531,21 @@ We categorize the changes as follows: * The Mesos agent now requires the new `AGENT_DRAINING` feature. This capability is set by default, but if the `--agent_features` flag is specified explicitly, `AGENT_DRAINING` must be included. + * A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The isolator supports setting of the `no_new_privs` bit in the container, preventing tasks from acquiring additional privileges. + * A new [`--docker_ignore_runtime`](configuration/agent.md#docker_ignore_runtime) flag has been added. This causes the agent to ignore any runtime configuration present in Docker images. + * A new libprocess TLS flag `--hostname_validation_scheme` along with the corresponding environment variable `LIBPROCESS_SSL_HOSTNAME_VALIDATION_SCHEME` has been added. Using this flag, users can configure the way libprocess performs hostname validation for TLS connections. See [`docs/ssl`](ssl.md) for details. + * The semantics of the libprocess environment variables `LIBPROCESS_SSL_VERIFY_CERT` and `LIBPROCESS_SSL_REQUIRE_CERT` have been slightly updated such that the former now only applies to client-mode and the latter only to server-mode connections. As part of this re-adjustment, the following two changes have been introduced that might require changes for operators running Mesos in unusual TLS configurations. @@ -548,8 +558,19 @@ We categorize the changes as follows: the `LIBPROCESS_SSL_REQUIRE_CERT` option is set to true. + * The Mesos containerizer now supports configurable IPC namespace and /dev/shm. Container can be configured to have a private IPC namespace and /dev/shm or share them from its parent via the field `LinuxInfo.ipc_mode`, and the size of its private /dev/shm is also configurable via the field `LinuxInfo.shm_size`. Operators can control whether it is allowed to share host's IPC namespace and /dev/shm with top level containers via the agent flag `--disallow_sharing_agent_ipc_namespace`, and s [...] + + +* The `SET_QUOTA` and `REMOVE QUOTA` master calls are deprecated in favor of a new `UPDATE_QUOTA` master call. + + + +* Prior to Mesos 1.9, the quota related APIs only exposed quota "guarantees" which ensured a minimum amount of resources would be available to a role. Setting guarantees also set implicit quota limits. In Mesos 1.9+, quota limits are now exposed directly. + * Quota guarantees are now deprecated in favor of using only quota limits. Enforcement of quota guarantees required that Mesos holds back enough resources to meet all of the unsatisfied quota guarantees. Since Mesos is moving towards an optimistic offer model (to improve multi-role / multi- scheduler scalability, see MESOS-1607), it will become no longer possible to enforce quota guarantees by holding back resources. In such a model, quota limits are simple to enforce, but quota guaran [...] + * For these reasons, quota guarantees, while still functional in Mesos 1.9, are now deprecated. A combination of limits and priority based preemption will be simpler in an optimistic offer model. + ## Upgrading from 1.7.x to 1.8.x ##
[mesos] branch master updated: Fixed formatting in the upgrade docs.
This is an automated email from the ASF dual-hosted git repository. grag pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git The following commit(s) were added to refs/heads/master by this push: new e5bf1b6 Fixed formatting in the upgrade docs. e5bf1b6 is described below commit e5bf1b61ec140f70ad90d522dc37a4ea82554221 Author: Greg Mann AuthorDate: Fri Aug 30 11:48:18 2019 -0300 Fixed formatting in the upgrade docs. --- docs/upgrades.md | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/upgrades.md b/docs/upgrades.md index d36a9a4..d745752 100644 --- a/docs/upgrades.md +++ b/docs/upgrades.md @@ -522,21 +522,22 @@ We categorize the changes as follows: - * A new `DRAINING` state has been added to Mesos agents. Once an agent is draining, all tasks running on that agent are gracefully -killed and no offers for that agent are sent to schedulers, preventing the launching of new tasks. -Operators can put an agent into `DRAINING` state by using the `DRAIN_AGENT` operator API call. -See [`docs/maintenance`](maintenance.md) for details. +* A new `DRAINING` state has been added to Mesos agents. Once an agent is draining, all tasks running on that agent are gracefully + killed and no offers for that agent are sent to schedulers, preventing the launching of new tasks. + Operators can put an agent into `DRAINING` state by using the `DRAIN_AGENT` operator API call. + See [`docs/maintenance`](maintenance.md) for details. + * The Mesos agent now requires the new `AGENT_DRAINING` feature. This capability is set by default, but if the `--agent_features` flag is specified explicitly, `AGENT_DRAINING` must be included. - * A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The isolator supports setting of the `no_new_privs` bit in the container, preventing tasks from acquiring additional privileges. +* A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The isolator supports setting of the `no_new_privs` bit in the container, preventing tasks from acquiring additional privileges. - * A new [`--docker_ignore_runtime`](configuration/agent.md#docker_ignore_runtime) flag has been added. This causes the agent to ignore any runtime configuration present in Docker images. +* A new [`--docker_ignore_runtime`](configuration/agent.md#docker_ignore_runtime) flag has been added. This causes the agent to ignore any runtime configuration present in Docker images.
[mesos-site] branch asf-site updated: Updated the website built from mesos SHA: 93963c46b.
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/mesos-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 260edef Updated the website built from mesos SHA: 93963c46b.
260edef is described below
commit 260edef6514c6ea8a77856e540a8d3b503d43083
Author: jenkins
AuthorDate: Fri Aug 30 13:51:45 2019 +
Updated the website built from mesos SHA: 93963c46b.
---
content/documentation/latest/configuration/agent/index.html | 7 ---
content/documentation/latest/upgrades/index.html| 4
content/documentation/upgrades/index.html | 4
3 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/content/documentation/latest/configuration/agent/index.html
b/content/documentation/latest/configuration/agent/index.html
index efcaafd..2d8be42 100644
--- a/content/documentation/latest/configuration/agent/index.html
+++ b/content/documentation/latest/configuration/agent/index.html
@@ -195,8 +195,8 @@ Example:
JSON representation of agent features to whitelist. We always require
-'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT', and
-'AGENT_OPERATION_FEEDBACK'.
+'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT',
+'AGENT_OPERATION_FEEDBACK', and 'AGENT_DRAINING'.
Example:
@@ -205,7 +205,8 @@ Example:
{"type": "MULTI_ROLE"},
{"type": "HIERARCHICAL_ROLE"},
{"type": "RESERVATION_REFINEMENT"},
-{"type": "AGENT_OPERATION_FEEDBACK"}
+{"type": "AGENT_OPERATION_FEEDBACK"},
+{"type": "AGENT_DRAINING"}
]
}
diff --git a/content/documentation/latest/upgrades/index.html
b/content/documentation/latest/upgrades/index.html
index 6d62b16..58f3395 100644
--- a/content/documentation/latest/upgrades/index.html
+++ b/content/documentation/latest/upgrades/index.html
@@ -165,6 +165,7 @@ R Removed feature/behavior
A docker_ignore_runtime
A disallow_sharing_agent_ipc_namespace
A default_container_shm_size
+ C agent_features
@@ -633,6 +634,9 @@ See docs/maintenance
+
+* The Mesos agent now requires the new AGENT_DRAINING feature.
This capability is set by default, but if the --agent_features
flag is specified explicitly, AGENT_DRAINING must be included.
+
diff --git a/content/documentation/upgrades/index.html
b/content/documentation/upgrades/index.html
index c519fe1..cc227e8 100644
--- a/content/documentation/upgrades/index.html
+++ b/content/documentation/upgrades/index.html
@@ -165,6 +165,7 @@ R Removed feature/behavior
A docker_ignore_runtime
A disallow_sharing_agent_ipc_namespace
A default_container_shm_size
+ C agent_features
@@ -633,6 +634,9 @@ See docs/maintenance
+
+* The Mesos agent now requires the new AGENT_DRAINING feature.
This capability is set by default, but if the --agent_features
flag is specified explicitly, AGENT_DRAINING must be included.
+
[mesos] branch 1.9.x updated: Updated docs for the AGENT_DRAINING capability.
This is an automated email from the ASF dual-hosted git repository.
grag pushed a commit to branch 1.9.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/1.9.x by this push:
new 091f193 Updated docs for the AGENT_DRAINING capability.
091f193 is described below
commit 091f193685142e5187a4f84d55ed1e28ac34749c
Author: Greg Mann
AuthorDate: Fri Aug 30 10:23:10 2019 -0300
Updated docs for the AGENT_DRAINING capability.
Review: https://reviews.apache.org/r/71405/
---
docs/configuration/agent.md | 7 ---
docs/upgrades.md| 4
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/docs/configuration/agent.md b/docs/configuration/agent.md
index 760d22b..91e38c2 100644
--- a/docs/configuration/agent.md
+++ b/docs/configuration/agent.md
@@ -92,8 +92,8 @@ Example:
JSON representation of agent features to whitelist. We always require
-'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT', and
-'AGENT_OPERATION_FEEDBACK'.
+'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT',
+'AGENT_OPERATION_FEEDBACK', and 'AGENT_DRAINING'.
Example:
@@ -102,7 +102,8 @@ Example:
{"type": "MULTI_ROLE"},
{"type": "HIERARCHICAL_ROLE"},
{"type": "RESERVATION_REFINEMENT"},
-{"type": "AGENT_OPERATION_FEEDBACK"}
+{"type": "AGENT_OPERATION_FEEDBACK"},
+{"type": "AGENT_DRAINING"}
]
}
diff --git a/docs/upgrades.md b/docs/upgrades.md
index ded4a8d..0345e22 100644
--- a/docs/upgrades.md
+++ b/docs/upgrades.md
@@ -61,6 +61,7 @@ We categorize the changes as follows:
A docker_ignore_runtime
A disallow_sharing_agent_ipc_namespace
A default_container_shm_size
+ C agent_features
@@ -520,6 +521,9 @@ We categorize the changes as follows:
Operators can put an agent into `DRAINING` state by using the
`DRAIN_AGENT` operator API call.
See [`docs/maintenance`](maintenance.md) for details.
+
+* The Mesos agent now requires the new `AGENT_DRAINING` feature. This
capability is set by default, but if the `--agent_features` flag is specified
explicitly, `AGENT_DRAINING` must be included.
+
* A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The
isolator supports setting of the `no_new_privs` bit in the container,
preventing tasks from acquiring additional privileges.
[mesos] branch master updated: Updated docs for the AGENT_DRAINING capability.
This is an automated email from the ASF dual-hosted git repository.
grag pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/master by this push:
new 93963c4 Updated docs for the AGENT_DRAINING capability.
93963c4 is described below
commit 93963c46b803f59554e12afe9c66605600c7f5b8
Author: Greg Mann
AuthorDate: Fri Aug 30 10:23:10 2019 -0300
Updated docs for the AGENT_DRAINING capability.
Review: https://reviews.apache.org/r/71405/
---
docs/configuration/agent.md | 7 ---
docs/upgrades.md| 4
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/docs/configuration/agent.md b/docs/configuration/agent.md
index 760d22b..91e38c2 100644
--- a/docs/configuration/agent.md
+++ b/docs/configuration/agent.md
@@ -92,8 +92,8 @@ Example:
JSON representation of agent features to whitelist. We always require
-'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT', and
-'AGENT_OPERATION_FEEDBACK'.
+'MULTI_ROLE', 'HIERARCHICAL_ROLE', 'RESERVATION_REFINEMENT',
+'AGENT_OPERATION_FEEDBACK', and 'AGENT_DRAINING'.
Example:
@@ -102,7 +102,8 @@ Example:
{"type": "MULTI_ROLE"},
{"type": "HIERARCHICAL_ROLE"},
{"type": "RESERVATION_REFINEMENT"},
-{"type": "AGENT_OPERATION_FEEDBACK"}
+{"type": "AGENT_OPERATION_FEEDBACK"},
+{"type": "AGENT_DRAINING"}
]
}
diff --git a/docs/upgrades.md b/docs/upgrades.md
index 31f4a19..d36a9a4 100644
--- a/docs/upgrades.md
+++ b/docs/upgrades.md
@@ -62,6 +62,7 @@ We categorize the changes as follows:
A docker_ignore_runtime
A disallow_sharing_agent_ipc_namespace
A default_container_shm_size
+ C agent_features
@@ -526,6 +527,9 @@ We categorize the changes as follows:
Operators can put an agent into `DRAINING` state by using the
`DRAIN_AGENT` operator API call.
See [`docs/maintenance`](maintenance.md) for details.
+
+* The Mesos agent now requires the new `AGENT_DRAINING` feature. This
capability is set by default, but if the `--agent_features` flag is specified
explicitly, `AGENT_DRAINING` must be included.
+
* A new [`linux/nnp`](isolators/linux-nnp.md) isolator has been added. The
isolator supports setting of the `no_new_privs` bit in the container,
preventing tasks from acquiring additional privileges.
[mesos] branch master updated: Added MESOS-9956 to the 1.9.0 CHANGELOG.
This is an automated email from the ASF dual-hosted git repository. bennoe pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git The following commit(s) were added to refs/heads/master by this push: new 7df2cff Added MESOS-9956 to the 1.9.0 CHANGELOG. 7df2cff is described below commit 7df2cff9d0951e3a7d2c45f1bd031e62be3d3ee1 Author: Benno Evers AuthorDate: Fri Aug 30 13:09:26 2019 +0200 Added MESOS-9956 to the 1.9.0 CHANGELOG. --- CHANGELOG | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG b/CHANGELOG index 6291d29..876c303 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -152,6 +152,7 @@ All Resolved Issues: * [MESOS-9934] - Master does not handle returning unreachable agents as draining/deactivated * [MESOS-9935] - The agent crashes after the disk du isolator supporting rootfs checks. * [MESOS-9952] - ExampleTest.DiskFullFramework is slow + * [MESOS-9956] - CSI plugins reporting duplicated volumes will crash the agent. ** Epic * [MESOS-9534] - CSI Spec v1.0 Support.
[mesos] branch 1.9.x updated: Added MESOS-9956 to the 1.9.0 CHANGELOG.
This is an automated email from the ASF dual-hosted git repository. bennoe pushed a commit to branch 1.9.x in repository https://gitbox.apache.org/repos/asf/mesos.git The following commit(s) were added to refs/heads/1.9.x by this push: new cacc0e7 Added MESOS-9956 to the 1.9.0 CHANGELOG. cacc0e7 is described below commit cacc0e7a629de4fb1e678d814b30fd716bcb29d7 Author: Benno Evers AuthorDate: Fri Aug 30 13:09:26 2019 +0200 Added MESOS-9956 to the 1.9.0 CHANGELOG. --- CHANGELOG | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG b/CHANGELOG index 6291d29..876c303 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -152,6 +152,7 @@ All Resolved Issues: * [MESOS-9934] - Master does not handle returning unreachable agents as draining/deactivated * [MESOS-9935] - The agent crashes after the disk du isolator supporting rootfs checks. * [MESOS-9952] - ExampleTest.DiskFullFramework is slow + * [MESOS-9956] - CSI plugins reporting duplicated volumes will crash the agent. ** Epic * [MESOS-9534] - CSI Spec v1.0 Support.
[mesos] branch 1.9.x updated: Gracefully handled duplicated volumes from non-conforming CSI plugins.
This is an automated email from the ASF dual-hosted git repository.
bennoe pushed a commit to branch 1.9.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/1.9.x by this push:
new b18ce53 Gracefully handled duplicated volumes from non-conforming CSI
plugins.
b18ce53 is described below
commit b18ce53fe8e49e6f030efe89e0976a9f72ad8b50
Author: Chun-Hung Hsiao
AuthorDate: Fri Aug 30 13:05:37 2019 +0200
Gracefully handled duplicated volumes from non-conforming CSI plugins.
If the SLRP uses a plugin that does not conform to the CSI spec and
reports duplicated volumes, the duplicate would be removed.
Review: https://reviews.apache.org/r/71414/
---
src/resource_provider/storage/provider.cpp | 78 +--
.../storage_local_resource_provider_tests.cpp | 109 +
2 files changed, 159 insertions(+), 28 deletions(-)
diff --git a/src/resource_provider/storage/provider.cpp
b/src/resource_provider/storage/provider.cpp
index f180af8..0a8dc26 100644
--- a/src/resource_provider/storage/provider.cpp
+++ b/src/resource_provider/storage/provider.cpp
@@ -673,8 +673,10 @@ Future
StorageLocalResourceProviderProcess::recover()
}
}
- LOG(INFO) << "Finished recovery for resource provider with type '"
-<< info.type() << "' and name '" << info.name() << "'";
+ LOG(INFO)
+<< "Recovered resources '" << totalResources << "' and "
+<< operations.size() << " operations for resource provider with type '"
+<< info.type() << "' and name '" << info.name() << "'";
state = DISCONNECTED;
@@ -1081,45 +1083,73 @@
StorageLocalResourceProviderProcess::getExistingVolumes()
return volumeManager->listVolumes()
.then(defer(self(), [=](const vector& volumeInfos) {
+ // If a volume is duplicated or a volume context has been changed by a
+ // non-conforming CSI plugin, we need to construct a resources conversion
+ // to remove the duplicate and update the metadata, so we maintain the
+ // resources to be removed and those to be added here.
+ Resources toRemove;
+ Resources toAdd;
+
// Since we only support "exclusive" (MOUNT or BLOCK) disks, there should
// be only one checkpointed resource for each volume ID.
hashmap checkpointedMap;
foreach (const Resource& resource, totalResources) {
if (resource.disk().source().has_id()) {
- CHECK(!checkpointedMap.contains(resource.disk().source().id()));
- checkpointedMap.put(resource.disk().source().id(), resource);
+ // If the checkpointed resources contain duplicated volumes because
of
+ // a non-conforming CSI plugin, remove the duplicate.
+ if (checkpointedMap.contains(resource.disk().source().id())) {
+LOG(WARNING) << "Removing duplicated volume '" << resource
+ << "' from the total resources";
+
+toRemove += resource;
+ } else {
+checkpointedMap.put(resource.disk().source().id(), resource);
+ }
}
}
// The "discovered" resources consist of RAW disk resources, one for each
// volume reported by the CSI plugin.
Resources discovered;
-
- // If any volume context has been changed by a non-conforming CSI plugin,
- // we need to construct a resources conversion to reflect the
- // corresponding metadata changes, so we maintain the resources to be
- // removed and those to be added here.
- Resources metadataToRemove;
- Resources metadataToAdd;
+ hashset discoveredVolumeIds;
foreach (const VolumeInfo& volumeInfo, volumeInfos) {
-Option profile;
-Option metadata = volumeInfo.context.empty()
+const Option profile =
+ checkpointedMap.contains(volumeInfo.id) &&
+ checkpointedMap.at(volumeInfo.id).disk().source().has_profile()
+? checkpointedMap.at(volumeInfo.id).disk().source().profile()
+: Option::none();
+
+const Option metadata = volumeInfo.context.empty()
? Option::none()
: convertStringMapToLabels(volumeInfo.context);
+const Resource resource = createRawDiskResource(
+info,
+volumeInfo.capacity,
+profile,
+vendor,
+volumeInfo.id,
+metadata);
+
+if (discoveredVolumeIds.contains(volumeInfo.id)) {
+ LOG(WARNING) << "Dropping duplicated volume '" << resource
+ << "' from the discovered resources";
+
+ continue;
+}
+
+discovered += resource;
+discoveredVolumeIds.insert(volumeInfo.id);
+
if (checkpointedMap.contains(volumeInfo.id)) {
const Resource& resource = checkpointedMap.at(volumeInfo.id);
- if (resource.disk().source().has_profile()) {
-
[mesos] branch master updated: Gracefully handled duplicated volumes from non-conforming CSI plugins.
This is an automated email from the ASF dual-hosted git repository.
bennoe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/master by this push:
new 43b86da Gracefully handled duplicated volumes from non-conforming CSI
plugins.
43b86da is described below
commit 43b86da531a889b1c4b1d7ca6acb2eb924ea01e1
Author: Chun-Hung Hsiao
AuthorDate: Fri Aug 30 13:04:22 2019 +0200
Gracefully handled duplicated volumes from non-conforming CSI plugins.
If the SLRP uses a plugin that does not conform to the CSI spec and
reports duplicated volumes, the duplicate would be removed.
Review: https://reviews.apache.org/r/71414/
---
src/resource_provider/storage/provider.cpp | 78 +--
.../storage_local_resource_provider_tests.cpp | 109 +
2 files changed, 159 insertions(+), 28 deletions(-)
diff --git a/src/resource_provider/storage/provider.cpp
b/src/resource_provider/storage/provider.cpp
index f180af8..0a8dc26 100644
--- a/src/resource_provider/storage/provider.cpp
+++ b/src/resource_provider/storage/provider.cpp
@@ -673,8 +673,10 @@ Future
StorageLocalResourceProviderProcess::recover()
}
}
- LOG(INFO) << "Finished recovery for resource provider with type '"
-<< info.type() << "' and name '" << info.name() << "'";
+ LOG(INFO)
+<< "Recovered resources '" << totalResources << "' and "
+<< operations.size() << " operations for resource provider with type '"
+<< info.type() << "' and name '" << info.name() << "'";
state = DISCONNECTED;
@@ -1081,45 +1083,73 @@
StorageLocalResourceProviderProcess::getExistingVolumes()
return volumeManager->listVolumes()
.then(defer(self(), [=](const vector& volumeInfos) {
+ // If a volume is duplicated or a volume context has been changed by a
+ // non-conforming CSI plugin, we need to construct a resources conversion
+ // to remove the duplicate and update the metadata, so we maintain the
+ // resources to be removed and those to be added here.
+ Resources toRemove;
+ Resources toAdd;
+
// Since we only support "exclusive" (MOUNT or BLOCK) disks, there should
// be only one checkpointed resource for each volume ID.
hashmap checkpointedMap;
foreach (const Resource& resource, totalResources) {
if (resource.disk().source().has_id()) {
- CHECK(!checkpointedMap.contains(resource.disk().source().id()));
- checkpointedMap.put(resource.disk().source().id(), resource);
+ // If the checkpointed resources contain duplicated volumes because
of
+ // a non-conforming CSI plugin, remove the duplicate.
+ if (checkpointedMap.contains(resource.disk().source().id())) {
+LOG(WARNING) << "Removing duplicated volume '" << resource
+ << "' from the total resources";
+
+toRemove += resource;
+ } else {
+checkpointedMap.put(resource.disk().source().id(), resource);
+ }
}
}
// The "discovered" resources consist of RAW disk resources, one for each
// volume reported by the CSI plugin.
Resources discovered;
-
- // If any volume context has been changed by a non-conforming CSI plugin,
- // we need to construct a resources conversion to reflect the
- // corresponding metadata changes, so we maintain the resources to be
- // removed and those to be added here.
- Resources metadataToRemove;
- Resources metadataToAdd;
+ hashset discoveredVolumeIds;
foreach (const VolumeInfo& volumeInfo, volumeInfos) {
-Option profile;
-Option metadata = volumeInfo.context.empty()
+const Option profile =
+ checkpointedMap.contains(volumeInfo.id) &&
+ checkpointedMap.at(volumeInfo.id).disk().source().has_profile()
+? checkpointedMap.at(volumeInfo.id).disk().source().profile()
+: Option::none();
+
+const Option metadata = volumeInfo.context.empty()
? Option::none()
: convertStringMapToLabels(volumeInfo.context);
+const Resource resource = createRawDiskResource(
+info,
+volumeInfo.capacity,
+profile,
+vendor,
+volumeInfo.id,
+metadata);
+
+if (discoveredVolumeIds.contains(volumeInfo.id)) {
+ LOG(WARNING) << "Dropping duplicated volume '" << resource
+ << "' from the discovered resources";
+
+ continue;
+}
+
+discovered += resource;
+discoveredVolumeIds.insert(volumeInfo.id);
+
if (checkpointedMap.contains(volumeInfo.id)) {
const Resource& resource = checkpointedMap.at(volumeInfo.id);
- if (resource.disk().source().has_profile()) {
-
