[17/17] mesos git commit: Renamed the `NewProfile` SLRP test and made it based on offers.

2018-07-13 Thread chhsiao
Renamed the `NewProfile` SLRP test and made it based on offers. The test is renamed to `ProfileAppeared` to reflect the fact that the SLRP passively observed a profile showing up, not actively adding a profile. The test is changed from intercepting `UpdateSlaveMessage`s to checking offers to

[01/17] mesos git commit: Made `UriDiskProfileAdaptor` be able to handle disappeared profiles.

2018-07-13 Thread chhsiao
Repository: mesos Updated Branches: refs/heads/master c2bdf07eb -> 73d1da089 Made `UriDiskProfileAdaptor` be able to handle disappeared profiles. Now the `UriDiskProfileAdaptor` module can handle disappeared profiles when polling the profile matrix, and performs a best-effort immutability

[12/17] mesos git commit: Fixed a bug in `TestCSIPlugin::DeleteVolume`.

2018-07-13 Thread chhsiao
Fixed a bug in `TestCSIPlugin::DeleteVolume`. The test CSI plugin mistakenly reduce the available capacity instead of expanding it when deleting a volume. This patch fixes this bug. Review: https://reviews.apache.org/r/67668 Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit:

[16/17] mesos git commit: Added a unit test for disappeared profiles.

2018-07-13 Thread chhsiao
Added a unit test for disappeared profiles. The `ROOT_ProfileDisappeared` tests if a `CREATE_VOLUME` consuming a RAW disk with a disappeared profile will be dropped, and if the disk space freed by a `DESTROY_VOLUME` destroying a volume with a disappeared profile will be recovered with a newly

[14/17] mesos git commit: Made disk type checks in SLRP hard assertions.

2018-07-13 Thread chhsiao
Made disk type checks in SLRP hard assertions. Since the disk types of the `CREATE_DISK` and `DESTROY_DISK` should have been validated by the master, we now use `CHECK`s instead of returning failures. Review: https://reviews.apache.org/r/67892 Project:

[08/17] mesos git commit: Allowed resources to be removed in the hierarchical allocator.

2018-07-13 Thread chhsiao
Allowed resources to be removed in the hierarchical allocator. When applying `DESTROY_VOLUME` or `DESTROY_BLOCK` on a CSI volume with a stale profile, the volume will be converted to an empty resource to avoid it to be offered to frameworks. This patch updates the hierarchical allocator to allow

[05/17] mesos git commit: Declined unwanted offers in `RetryOperationStatusUpdate*` SLRP tests.

2018-07-13 Thread chhsiao
Declined unwanted offers in `RetryOperationStatusUpdate*` SLRP tests. The two SLRP tests assume that SLRP will send out a RAW resource in its first `UPDATE_STATE` message, and expect that the test framework would receive an offer containing the RAW resource in its first offer. However this

[06/17] mesos git commit: Updated tests to adapt `CREATE_DISK` and `DESTROY_DISK` operations.

2018-07-13 Thread chhsiao
Updated tests to adapt `CREATE_DISK` and `DESTROY_DISK` operations. Review: https://reviews.apache.org/r/67864 Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/73d1da08 Tree:

[09/17] mesos git commit: Fixed a race between `UPDATE_STATE` and `UPDATE_OPERATION_STATUS`.

2018-07-13 Thread chhsiao
Fixed a race between `UPDATE_STATE` and `UPDATE_OPERATION_STATUS`. Since a resource provider and its operation status update manager run in different actors, the `UPDATE_OPERATION_STATUS` call of a completed operation may race with an `UPDATE_STATE` call. To deal with this race, the agent should

[04/17] mesos git commit: Reconciled storage pools when destroying volumes with stale profiles.

2018-07-13 Thread chhsiao
Reconciled storage pools when destroying volumes with stale profiles. The storage pools needs to be reconciled in the following two scenarios: 1. When there is a change in the set of known profiles. 2. When a volume/block of a disappeared profile is destroyed, because the disk space being

[13/17] mesos git commit: Updated Mesos implementation to process `CREATE_DISK` & `DESTROY_DISK`.

2018-07-13 Thread chhsiao
Updated Mesos implementation to process `CREATE_DISK` & `DESTROY_DISK`. This patch removes the code for `CREATE_VOLUME`, `DESTROY_VOLUME`, `CREATE_BLOCK` and `DESTROY_BLOCK` to adapt the new `CREATE_DISK` and `DESTROY_DISK` operations. Review: https://reviews.apache.org/r/67863 Project:

[15/17] mesos git commit: Made the `UriDiskProfileAdaptor` module configurable in SLRP tests.

2018-07-13 Thread chhsiao
Made the `UriDiskProfileAdaptor` module configurable in SLRP tests. This patch changes the `setupDiskProfileMapping` helper used in SLRP tests to `createDiskProfileMapping`, which returns a JSON string with a configurable profile name instead of writing the JSON into a fixed path. This gives us

[03/17] mesos git commit: Made SLRP be able to handle disappeared profiles.

2018-07-13 Thread chhsiao
Made SLRP be able to handle disappeared profiles. When profile disappears, SLRP will remove it from its profile metadata. Since we maintain an invariant that profiles of all accepted operations must exist, the assertions related to profile existence remain true. Review:

[10/17] mesos git commit: Allowed resources to be removed when updating the sorter.

2018-07-13 Thread chhsiao
Allowed resources to be removed when updating the sorter. This patch allows the allocator to remove resources when calling `Sorter::update`, so Mesos can remove resources that are no longer available through resource conversions. An example usage is that we can make `DESTROY_VOLUME` to convert a

[02/17] mesos git commit: Fixed a race condition in `UriDiskProfileAdaptorTests`.

2018-07-13 Thread chhsiao
Fixed a race condition in `UriDiskProfileAdaptorTests`. There was a race between `Clock::advance()` in the `FetchFromHTTP` test and `delay()` in `UriDiskProfileAdaptorProcess::_poll`. This patch avoids the race by enforcing an order between the dispatch of the `__poll` function (previously

[07/17] mesos git commit: Introduced `CREATE_DISK` and `DESTROY_DISK` operations.

2018-07-13 Thread chhsiao
Introduced `CREATE_DISK` and `DESTROY_DISK` operations. This patch introduces `CREATE_DISK` and `DESTROY_DISK` operations for conversion between RAW disk resaurces and MOUNT/BLOCK disk resources to replace `CREATE_VOLUME`/`DESTROY_VOLUME`/`CREATE_BLOCK`/`DESTROY_BLOCK`. This replacement involves

[5/5] mesos-site git commit: Updated the website built from mesos SHA: 13fb36e.

2018-07-13 Thread git-site-role
Updated the website built from mesos SHA: 13fb36e. Project: http://git-wip-us.apache.org/repos/asf/mesos-site/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos-site/commit/fbfc758a Tree: http://git-wip-us.apache.org/repos/asf/mesos-site/tree/fbfc758a Diff:

[2/5] mesos-site git commit: Updated the website built from mesos SHA: 13fb36e.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/fbfc758a/content/api/latest/c++/index.hhc -- diff --git a/content/api/latest/c++/index.hhc b/content/api/latest/c++/index.hhc index bf27fd3..fa75900 100644 ---

[4/5] mesos-site git commit: Updated the website built from mesos SHA: 13fb36e.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/fbfc758a/content/api/latest/c++/classes.html -- diff --git a/content/api/latest/c++/classes.html b/content/api/latest/c++/classes.html index dc9ed8c..65ce22d 100644 ---

[1/5] mesos-site git commit: Updated the website built from mesos SHA: 13fb36e.

2018-07-13 Thread git-site-role
Repository: mesos-site Updated Branches: refs/heads/asf-site 3fc115a8e -> fbfc758ad http://git-wip-us.apache.org/repos/asf/mesos-site/blob/fbfc758a/content/sitemap.xml -- diff --git a/content/sitemap.xml b/content/sitemap.xml

mesos git commit: Revert "Optimized the generation of metrics snapshots."

2018-07-13 Thread grag
Repository: mesos Updated Branches: refs/heads/master 133379439 -> 13fb36ec0 Revert "Optimized the generation of metrics snapshots." This reverts commit 42becf22f07831ef8c3d4fbc8c1b80eb9d5b959a. Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit:

[3/5] mesos-site git commit: Updated the website built from mesos SHA: 13fb36e.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/fbfc758a/content/api/latest/c++/hierarchy.html -- diff --git a/content/api/latest/c++/hierarchy.html b/content/api/latest/c++/hierarchy.html index 1061513..86bfdad 100644 ---

[11/17] mesos git commit: Added a `TestDiskProfileServer` helper.

2018-07-13 Thread chhsiao
Added a `TestDiskProfileServer` helper. This helper class will be used to test functionalities related to profile changes. Review: https://reviews.apache.org/r/67666 Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/0e90d7e2

[08/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.CreateDisk.Builder.html -- diff --git

[17/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/src_2master_2master_8hpp_source.html -- diff --git a/content/api/latest/c++/src_2master_2master_8hpp_source.html

[32/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
Updated the website built from mesos SHA: 73d1da0. Project: http://git-wip-us.apache.org/repos/asf/mesos-site/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos-site/commit/dd06fd2f Tree: http://git-wip-us.apache.org/repos/asf/mesos-site/tree/dd06fd2f Diff:

[16/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/src_2master_2metrics_8hpp_source.html -- diff --git a/content/api/latest/c++/src_2master_2metrics_8hpp_source.html

[02/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.GrowVolume.html -- diff --git

[11/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/index-all.html -- diff --git a/content/api/latest/java/index-all.html b/content/api/latest/java/index-all.html index f0da54d..e62144c 100644

[26/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/hierarchy.html -- diff --git a/content/api/latest/c++/hierarchy.html b/content/api/latest/c++/hierarchy.html index 86bfdad..4f79fcf 100644 ---

[30/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/classmesos_1_1internal_1_1master_1_1Master-members.html -- diff --git

[04/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.DestroyDisk.Builder.html -- diff --git

[27/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/functions_func_p.html -- diff --git a/content/api/latest/c++/functions_func_p.html b/content/api/latest/c++/functions_func_p.html index

[06/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.CreateVolume.html -- diff --git

[07/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.CreateDiskOrBuilder.html -- diff --git

[21/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/mock__csi__plugin_8hpp.html -- diff --git a/content/api/latest/c++/mock__csi__plugin_8hpp.html

[31/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/classes.html -- diff --git a/content/api/latest/c++/classes.html b/content/api/latest/c++/classes.html index 65ce22d..e281d81 100644 ---

[03/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.DestroyOrBuilder.html -- diff --git

[25/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/http__connection_8hpp_source.html -- diff --git a/content/api/latest/c++/http__connection_8hpp_source.html

[22/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/master_2validation_8hpp_source.html -- diff --git a/content/api/latest/c++/master_2validation_8hpp_source.html

[13/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/uri__disk__profile__adaptor_8hpp_source.html -- diff --git a/content/api/latest/c++/uri__disk__profile__adaptor_8hpp_source.html

[14/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/structprocess_1_1EventConsumer.html -- diff --git a/content/api/latest/c++/structprocess_1_1EventConsumer.html

[10/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.Builder.html -- diff --git

[12/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/v0__v1executor_8hpp_source.html -- diff --git a/content/api/latest/c++/v0__v1executor_8hpp_source.html

[20/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/slave_2validation_8hpp_source.html -- diff --git a/content/api/latest/c++/slave_2validation_8hpp_source.html

[01/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
Repository: mesos-site Updated Branches: refs/heads/asf-site fbfc758ad -> dd06fd2ff http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.OperationOrBuilder.html

[5/5] mesos git commit: Updated pylint usage in mesos-style.py to be less verbose.

2018-07-13 Thread klueska
Updated pylint usage in mesos-style.py to be less verbose. We started using Pylint 1.9 a few days ago to lint `.py` files, and this version of pylint always prints a score after linting is complete. To make the output less verbose, we now use the option `--score=n` when using pylint in

[4/5] mesos git commit: Updated tox usage in mesos-style.py to run directly from virtualenv.

2018-07-13 Thread klueska
Updated tox usage in mesos-style.py to run directly from virtualenv. We were already doing this in the ptyhon2 mesos-style.py script. This commit adds the same support to the python3 version of the script. Review: https://reviews.apache.org/r/67906/ Project:

[3/5] mesos git commit: Fixed virtualenv management in support directory.

2018-07-13 Thread klueska
Fixed virtualenv management in support directory. The switch from Python 2 to Python 3 creates problems with managing the virtual environment in the support directory if a developer regularly toggles between Python 2 and 3. For example, we want the virtual environment to always use the same

[2/5] mesos git commit: Fixed linting errors that appear with the recently updated pylint.

2018-07-13 Thread klueska
Fixed linting errors that appear with the recently updated pylint. As part of this, we needed to remove some unnecessary delegate to 'super' calls that are legal in python2 but not in python3. That said, they are also unnecessary in python2 so we remove them in both cases. Specifically, we

[1/5] mesos git commit: Fixed `build-virtualenv` to correctly clear virtual environment.

2018-07-13 Thread klueska
Repository: mesos Updated Branches: refs/heads/master 13fb36ec0 -> c2bdf07eb Fixed `build-virtualenv` to correctly clear virtual environment. Running `build-virtualenv` with Python 3 followed by running it again with Python 2 leaves the directory with both versions of Python in the

mesos git commit: Added Gastón Kleiman to the list of committers.

2018-07-13 Thread gaston
Repository: mesos Updated Branches: refs/heads/master 73d1da089 -> a11a6a3d8 Added Gastón Kleiman to the list of committers. Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/a11a6a3d Tree:

[28/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/disk__profile__server_8hpp_source.html -- diff --git a/content/api/latest/c++/disk__profile__server_8hpp_source.html

[24/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/index.hhk -- diff --git a/content/api/latest/c++/index.hhk b/content/api/latest/c++/index.hhk index 71445eb..476504f 100644 ---

[15/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/src_2tests_2mesos_8hpp_source.html -- diff --git a/content/api/latest/c++/src_2tests_2mesos_8hpp_source.html

[05/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.DestroyBlock.Builder.html -- diff --git

[18/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/sorter_8hpp_source.html -- diff --git a/content/api/latest/c++/sorter_8hpp_source.html b/content/api/latest/c++/sorter_8hpp_source.html index

[09/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/java/org/apache/mesos/Protos.Offer.Operation.Create.Builder.html -- diff --git

[19/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/slave_8hpp_source.html -- diff --git a/content/api/latest/c++/slave_8hpp_source.html b/content/api/latest/c++/slave_8hpp_source.html index

[29/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/classmesos_1_1internal_1_1tests_1_1TestDiskProfileServerProcess-members.html -- diff --git

[23/32] mesos-site git commit: Updated the website built from mesos SHA: 73d1da0.

2018-07-13 Thread git-site-role
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/dd06fd2f/content/api/latest/c++/index.hhp -- diff --git a/content/api/latest/c++/index.hhp b/content/api/latest/c++/index.hhp index 0a65ee8..39f4437 100644 ---

[1/2] mesos-site git commit: Updated the website built from mesos SHA: a11a6a3.

2018-07-13 Thread git-site-role
Repository: mesos-site Updated Branches: refs/heads/asf-site dd06fd2ff -> d8ccd4cd0 http://git-wip-us.apache.org/repos/asf/mesos-site/blob/d8ccd4cd/content/sitemap.xml -- diff --git a/content/sitemap.xml b/content/sitemap.xml

[2/2] mesos-site git commit: Updated the website built from mesos SHA: a11a6a3.

2018-07-13 Thread git-site-role
Updated the website built from mesos SHA: a11a6a3. Project: http://git-wip-us.apache.org/repos/asf/mesos-site/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos-site/commit/d8ccd4cd Tree: http://git-wip-us.apache.org/repos/asf/mesos-site/tree/d8ccd4cd Diff: