Re: Review Request 69818: Added offer filters to static framework configuration.

[Benjamin Bannier]

> On Jan. 29, 2019, 9:14 p.m., Benjamin Mahler wrote:
> > src/common/resource_quantities.cpp
> > Lines 82-90 (patched)
> > 
> >
> > Can we add some validation somewhere to ensure that negative value 
> > scalars are not provided?

https://reviews.apache.org/r/69862/


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69818/#review212419
---


On Jan. 30, 2019, 4:04 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69818/
> ---
> 
> (Updated Jan. 30, 2019, 4:04 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9523
> https://issues.apache.org/jira/browse/MESOS-9523
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds a new field to `FrameworkInfo` which can be used by
> frameworks to configure static, per-role resource requirements. We
> currently support specifying minimal allocatable resources which can
> e.g., be interpreted by allocators to filter out resources not relevant
> to frameworks, or make allocations below globally configured minimal
> allocatable resources for particular frameworks.
> 
> 
> Diffs
> -
> 
>   include/mesos/mesos.proto 3477b7db4dc64187aa55e8e4feb66a62f7951429 
>   include/mesos/v1/mesos.proto 692940bc862e61b9899aa159f29b662f3aade033 
>   src/common/resource_quantities.hpp 11eb426104577b7977c2307df3e4917085cd 
>   src/common/resource_quantities.cpp 320983929cd7d14973c4b98d6ed5338de690ff5f 
> 
> 
> Diff: https://reviews.apache.org/r/69818/diff/4/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69872: Accounted for possible lack of OperationID in RP responses.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69872/#review212476
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['69872']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2839/mesos-review-69872

Relevant logs:

- 
[mesos-tests.log](http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2839/mesos-review-69872/logs/mesos-tests.log):

```
I0131 05:18:37.143903 13316 master.cpp:1269] Agent 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-S0 at slave(477)@192.10.1.6:63918 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net) disconnected
I0131 05:18:37.143903 13316 master.cpp:3272] Disconnecting agent 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-S0 at slave(477)@192.10.1.6:63918 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0131 05:18:37.145030 13316 master.cpp:3291] Deactivating agent 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-S0 at slave(477)@192.10.1.6:63918 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0131 05:18:37.145030 16648 hierarchical.cpp:358] Removed framework 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-
I0131 05:18:37.145030 16648 hierarchical.cpp:793] Agent 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-S0 deactivated
I0131 05:18:37.146067 16648 containerizer.cpp:2477] Destroying container 
b1f2046d-0211-42be-abdc-bee77acf2ff0 in RUNNING state
I0131 05:18:37.146067 16648 containerizer.cpp:3144] Transitioning the state of 
container b1f2046d-0211-42be-abdc-bee77acf2ff0 from RUNNING to DESTROYING
I0131 05:18:37.146898 16648 launcher.cpp:161] Asked to destroy container 
b1f2046d-0211-42be-abdc-bee77acf2ff0
W0131 05:18:37.147900 28488 process.cpp:1423] Failed to recv on socket 
WindowsFD::Type::SOCKET=10340 to peer '192.10.1.6:49424': IO [   OK ] 
IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (688 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (705 ms total)

[--] Global test environment tear-down
[==] 1095 tests from 104 test cases ran. (551625 ms total)
[  PASSED  ] 1093 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] DockerFetcherPluginTest.INTERNET_CURL_FetchBlob
[  FAILED  ] DockerFetcherPluginTest.INTERNET_CURL_InvokeFetchByName

 2 FAILED TESTS
  YOU HAVE 231 DISABLED TESTS

failed with error code: The specified network name is no longer available.

W0131 05:18:37.148897 28488 process.cpp:838] Failed to recv on socket 
WindowsFD::Type::SOCKET=10272 to peer '192.10.1.6:49425': IO failed with error 
code: The specified network name is no longer available.

I0131 05:18:37.246955 13316 containerizer.cpp:2983] Container 
b1f2046d-0211-42be-abdc-bee77acf2ff0 has exited
I0131 05:18:37.275897 29344 master.cpp:1109] Master terminating
I0131 05:18:37.276898 20436 hierarchical.cpp:644] Removed agent 
2ee4ad16-252f-43d8-9cfc-3ad5273bef3f-S0
I0131 05:18:37.660959 28488 process.cpp:927] Stopped the socket accept loop
```

- Mesos Reviewbot Windows


On Jan. 31, 2019, 2:44 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69872/
> ---
> 
> (Updated Jan. 31, 2019, 2:44 a.m.)
> 
> 
> Review request for mesos, Benno Evers, Gastón Kleiman, and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Operations sent to a resource provider may include an OperationID
> in addition to an Operation UUID.  The UUID is sufficient to uniquely
> identify the operation, so it is possible for the resource provider
> to omit the OperationID in its responses to the agent.
> 
> This commit lets the agent fill in any missing ID before updates
> are acted upon, such as being sent to the master.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 0182dd2ca326723e96eef8c072696ad3c873de0b 
> 
> 
> Diff: https://reviews.apache.org/r/69872/diff/1/
> 
> 
> Testing
> ---
> 
> Ran the test here (https://reviews.apache.org/r/69869/diff/1) to get past 
> line 11692.
> 
> I can probably repurpose that test, if we don't have a CREATE_DISK + feedback 
> test already.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 69871: Added more documentation for operation feedback.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69871/#review212475
---



PASS: Mesos patch 69871 was successfully built and tested.

Reviews applied: `['69871']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2838/mesos-review-69871

- Mesos Reviewbot Windows


On Jan. 31, 2019, 10:11 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69871/
> ---
> 
> (Updated Jan. 31, 2019, 10:11 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and James 
> DeFelice.
> 
> 
> Bugs: MESOS-9477
> https://issues.apache.org/jira/browse/MESOS-9477
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added more documentation for operation feedback.
> 
> 
> Diffs
> -
> 
>   docs/high-availability-framework-guide.md 
> 1bf58e166c76af9ef9d7182d9d4e413bc02e253c 
>   docs/reconciliation.md bfb117c11969f97127ef2c55321191232814a1c2 
>   docs/scheduler-http-api.md 84f6187943b60d4472a4b7e0c667678ab055e482 
> 
> 
> Diff: https://reviews.apache.org/r/69871/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 69871: Added more documentation for operation feedback.

[James DeFelice]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69871/#review212473
---




docs/high-availability-framework-guide.md
Lines 254 (patched)


What about OPERATION_UNKNOWN, where does that fit in here?


- James DeFelice


On Jan. 31, 2019, 2:11 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69871/
> ---
> 
> (Updated Jan. 31, 2019, 2:11 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and James 
> DeFelice.
> 
> 
> Bugs: MESOS-9477
> https://issues.apache.org/jira/browse/MESOS-9477
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added more documentation for operation feedback.
> 
> 
> Diffs
> -
> 
>   docs/high-availability-framework-guide.md 
> 1bf58e166c76af9ef9d7182d9d4e413bc02e253c 
>   docs/reconciliation.md bfb117c11969f97127ef2c55321191232814a1c2 
>   docs/scheduler-http-api.md 84f6187943b60d4472a4b7e0c667678ab055e482 
> 
> 
> Diff: https://reviews.apache.org/r/69871/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 69872: Accounted for possible lack of OperationID in RP responses.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69872/#review212474
---


Fix it, then Ship it!





src/slave/slave.cpp
Lines 8119 (patched)


Let's do the injection before passing it into `updateOperation`.


- Chun-Hung Hsiao


On Jan. 31, 2019, 2:44 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69872/
> ---
> 
> (Updated Jan. 31, 2019, 2:44 a.m.)
> 
> 
> Review request for mesos, Benno Evers, Gastón Kleiman, and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Operations sent to a resource provider may include an OperationID
> in addition to an Operation UUID.  The UUID is sufficient to uniquely
> identify the operation, so it is possible for the resource provider
> to omit the OperationID in its responses to the agent.
> 
> This commit lets the agent fill in any missing ID before updates
> are acted upon, such as being sent to the master.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 0182dd2ca326723e96eef8c072696ad3c873de0b 
> 
> 
> Diff: https://reviews.apache.org/r/69872/diff/1/
> 
> 
> Testing
> ---
> 
> Ran the test here (https://reviews.apache.org/r/69869/diff/1) to get past 
> line 11692.
> 
> I can probably repurpose that test, if we don't have a CREATE_DISK + feedback 
> test already.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Review Request 69872: Accounted for possible lack of OperationID in RP responses.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69872/
---

Review request for mesos, Benno Evers, Gastón Kleiman, and Greg Mann.


Repository: mesos


Description
---

Operations sent to a resource provider may include an OperationID
in addition to an Operation UUID.  The UUID is sufficient to uniquely
identify the operation, so it is possible for the resource provider
to omit the OperationID in its responses to the agent.

This commit lets the agent fill in any missing ID before updates
are acted upon, such as being sent to the master.


Diffs
-

  src/slave/slave.cpp 0182dd2ca326723e96eef8c072696ad3c873de0b 


Diff: https://reviews.apache.org/r/69872/diff/1/


Testing
---

Ran the test here (https://reviews.apache.org/r/69869/diff/1) to get past line 
11692.

I can probably repurpose that test, if we don't have a CREATE_DISK + feedback 
test already.


Thanks,

Joseph Wu

Re: Review Request 69869: [WIP] Added test for tearing down frameworks while creating disks.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69869/#review212472
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['69869']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2837/mesos-review-69869

Relevant logs:

- 
[mesos-tests.log](http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2837/mesos-review-69869/logs/mesos-tests.log):

```
W0131 02:42:53.282217 27312 slave.cpp:3934] Ignoring shutdown framework 
4f25b998-2b3d-46b6-92d8-f59210e156af- because it is terminating
I0131 02:42:53.284219 27700 master.cpp:1269] Agent 
4f25b998-2b3d-46b6-92d8-f59210e156af-S0 at slave(478)@192.10.1.6:59648 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net) disconnected
I0131 02:42:53.284219 27700 master.cpp:3272] Disconnecting agent 
4f25b998-2b3d-46b6-92d8-f59210e156af-S0 at slave(478)@192.10.1.6:59648 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0131 02:42:53.284219 27700 master.cpp:3291] Deactivating agent 
4f25b998-2b3d-46b6-92d8-f59210e156af-S0 at slave(478)@192.10.1.6:59648 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0131 02:42:53.284219 27524 hierarchical.cpp:358] Removed framework 
4f25b998-2b3d-46b6-92d8-f59210e156af-
I0131 02:42:53.285212 27524 hierarchical.cpp:793] Agent 
4f25b998-2b3d-46b6-92d8-f59210e156af-S0 deactivated
I0131 02:42:53.286213 27504 containerizer.cpp:2477] Destroying container 
60d4eba9-ceae-4c21-bd7e-755ddb5d1605 in RUNNING state
I0131 02:42:53.286213 27504 containerizer.cpp:3144] Transitioning the state of 
container 60d4eba9-ceae-4c21-bd7e-755ddb5d1605 from RUNNING to DESTROYING
I0131 02:42:53.287214 27504 launcher.cpp:161] Asked to destroy container 
60d4eba9-ceae-4c21-bd7e-755ddb5d1605
W0131 02:42:53.288219 26296 process.cpp:838] Failed to recv on socket 
WindowsFD::Type::SOCKET=6384 to peer '192.10.1.6:61562': IO failed with error 
code: The specified network name is no longer available.

W0131 02:42:53.288219 26296 process.cpp:1423] Failed to recv on socket 
WindowsFD::Type::SOCKET=6528 to peer '192.10.1.6:61561': IO failed with error 
code: The specified network name is no longer available.

I0131 02:42:53.363895 27700 containerizer.cpp:2983] Container 
60d4eba9-ceae-4c21-bd7e-755ddb5d1605 has exited
I0131 02:42:53.392890 26164 master.cpp:1109] Master terminating
I0131 02:42:53.394879 27504 hierarchical.cpp:644] Removed agent 4f25b99[   
OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (686 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (703 ms total)

[--] Global test environment tear-down
[==] 1096 tests from 104 test cases ran. (519202 ms total)
[  PASSED  ] 1095 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] SlaveTest.FrameworkTeardownBeforeTerminalOperationStatusUpdate

 1 FAILED TEST
  YOU HAVE 231 DISABLED TESTS

8-2b3d-46b6-92d8-f59210e156af-S0
I0131 02:42:53.800878 26296 process.cpp:927] Stopped the socket accept loop
```

- Mesos Reviewbot Windows


On Jan. 31, 2019, 1:38 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69869/
> ---
> 
> (Updated Jan. 31, 2019, 1:38 a.m.)
> 
> 
> Review request for mesos, Benno Evers, Gastón Kleiman, and Greg Mann.
> 
> 
> Bugs: MESOS-9542
> https://issues.apache.org/jira/browse/MESOS-9542
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The CREATE_DISK and DESTROY_DISK operations are "non-speculative"
> operations, which means the master must wait for the operations to
> complete successfully before the master can update its resources.
> Because the master must wait to update the results of non-speculative
> operations, it is possible for the framework making the
> CREATE/DESTROY_DISK to be torn down before the operation completes.
> 
> This commit adds a test to make sure the master can gracefully handle
> such a case.
> 
> 
> Diffs
> -
> 
>   src/tests/slave_tests.cpp 22a0295086ae4f4ec26df00a0e077eecfa27f1fb 
> 
> 
> Diff: https://reviews.apache.org/r/69869/diff/1/
> 
> 
> Testing
> ---
> 
> The test does not currently pass, as it is meant to be a regression
> test for MESOS-9542 (a bug which has not been started yet).
> 
> src/mesos-tests 
> --gtest_filter="SlaveTest.FrameworkTeardownBeforeTerminalOperationStatusUpdate"
>  --verbose
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 69866: Updated SLRP test `ProfileDisappeared` to request operation feedbacks.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69866/#review212471
---



Patch looks great!

Reviews applied: [69858, 69866]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Jan. 31, 2019, 7:41 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69866/
> ---
> 
> (Updated Jan. 31, 2019, 7:41 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.
> 
> 
> Bugs: MESOS-9537
> https://issues.apache.org/jira/browse/MESOS-9537
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch updates `StorageLocalResourceProviderTest.ProfileDisappeared`
> to use the v1 scheduler API to request operation feedbacks, so
> MESOS-9537 would be triggered when an outstanding `UPDATE_STATE` call
> from the resource provider races with an offer operation.
> 
> 
> Diffs
> -
> 
>   src/tests/mesos.hpp c2a5e5531de7498241e537ef1699e1a5e669b550 
>   src/tests/storage_local_resource_provider_tests.cpp 
> e8ed20f818ed7f1a3ce15758ea3c366520443377 
> 
> 
> Diff: https://reviews.apache.org/r/69866/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> Ran the test ~1000 times.
> 
> This test will fail without r/69858:
> ```
> E0130 22:48:02.790385 24040 provider.cpp:2903] Failed to update status of 
> operation (uuid: 2bf6813c-bde9-4c0e-8831-a779f5dc50ac): Mismatched framework 
> ID for operation status update OPERATION_DROPPED (Status UUID: 
> 8a625c40-96e8-46e2-8050-69843032969a) for operation UUID 
> 2bf6813c-bde9-4c0e-8831-a779f5dc50ac on agent 
> bfa6e505-54b7-4bb8-a8aa-99fb12291fb1-S0 (expected 
> bfa6e505-54b7-4bb8-a8aa-99fb12291fb1- got no framework ID)
> I0130 22:48:02.791590 24042 manager.cpp:163] Terminating resource provider 
> 765391c2-9485-4614-a6e5-45e67ef7a92c
> ...
> ../../src/tests/storage_local_resource_provider_tests.cpp:1387: Failure
>   Expected: v1::OPERATION_FINISHED
>   Which is: OPERATION_FINISHED
> To be equal to: update->status().state()
>   Which is: OPERATION_ERROR
> ../../src/tests/storage_local_resource_provider_tests.cpp:1393: Failure
> Failed to wait 15secs for offers
> ...
> ../../src/tests/storage_local_resource_provider_tests.cpp:1361: Failure
> Actual function call count doesn't match EXPECT_CALL(*scheduler, offers(_, 
> v1::scheduler::OffersHaveAnyResource( std::bind(isStoragePool, lambda::_1, 
> "test2"...
>  Expected: to be called once
>Actual: never called - unsatisfied and active
> ```
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Review Request 69871: Added more documentation for operation feedback.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69871/
---

Review request for mesos, Benjamin Bannier, Gastón Kleiman, and James DeFelice.


Bugs: MESOS-9477
https://issues.apache.org/jira/browse/MESOS-9477


Repository: mesos


Description
---

Added more documentation for operation feedback.


Diffs
-

  docs/high-availability-framework-guide.md 
1bf58e166c76af9ef9d7182d9d4e413bc02e253c 
  docs/reconciliation.md bfb117c11969f97127ef2c55321191232814a1c2 
  docs/scheduler-http-api.md 84f6187943b60d4472a4b7e0c667678ab055e482 


Diff: https://reviews.apache.org/r/69871/diff/1/


Testing
---


Thanks,

Greg Mann

Review Request 69869: [WIP] Added test for tearing down frameworks while creating disks.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69869/
---

Review request for mesos, Benno Evers, Gastón Kleiman, and Greg Mann.


Bugs: MESOS-9542
https://issues.apache.org/jira/browse/MESOS-9542


Repository: mesos


Description
---

The CREATE_DISK and DESTROY_DISK operations are "non-speculative"
operations, which means the master must wait for the operations to
complete successfully before the master can update its resources.
Because the master must wait to update the results of non-speculative
operations, it is possible for the framework making the
CREATE/DESTROY_DISK to be torn down before the operation completes.

This commit adds a test to make sure the master can gracefully handle
such a case.


Diffs
-

  src/tests/slave_tests.cpp 22a0295086ae4f4ec26df00a0e077eecfa27f1fb 


Diff: https://reviews.apache.org/r/69869/diff/1/


Testing
---

The test does not currently pass, as it is meant to be a regression
test for MESOS-9542 (a bug which has not been started yet).

src/mesos-tests 
--gtest_filter="SlaveTest.FrameworkTeardownBeforeTerminalOperationStatusUpdate" 
--verbose


Thanks,

Joseph Wu

Re: Review Request 69866: Updated SLRP test `ProfileDisappeared` to request operation feedbacks.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69866/#review212469
---



PASS: Mesos patch 69866 was successfully built and tested.

Reviews applied: `['69858', '69866']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2836/mesos-review-69866

- Mesos Reviewbot Windows


On Jan. 30, 2019, 11:41 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69866/
> ---
> 
> (Updated Jan. 30, 2019, 11:41 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.
> 
> 
> Bugs: MESOS-9537
> https://issues.apache.org/jira/browse/MESOS-9537
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch updates `StorageLocalResourceProviderTest.ProfileDisappeared`
> to use the v1 scheduler API to request operation feedbacks, so
> MESOS-9537 would be triggered when an outstanding `UPDATE_STATE` call
> from the resource provider races with an offer operation.
> 
> 
> Diffs
> -
> 
>   src/tests/mesos.hpp c2a5e5531de7498241e537ef1699e1a5e669b550 
>   src/tests/storage_local_resource_provider_tests.cpp 
> e8ed20f818ed7f1a3ce15758ea3c366520443377 
> 
> 
> Diff: https://reviews.apache.org/r/69866/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> Ran the test ~1000 times.
> 
> This test will fail without r/69858:
> ```
> E0130 22:48:02.790385 24040 provider.cpp:2903] Failed to update status of 
> operation (uuid: 2bf6813c-bde9-4c0e-8831-a779f5dc50ac): Mismatched framework 
> ID for operation status update OPERATION_DROPPED (Status UUID: 
> 8a625c40-96e8-46e2-8050-69843032969a) for operation UUID 
> 2bf6813c-bde9-4c0e-8831-a779f5dc50ac on agent 
> bfa6e505-54b7-4bb8-a8aa-99fb12291fb1-S0 (expected 
> bfa6e505-54b7-4bb8-a8aa-99fb12291fb1- got no framework ID)
> I0130 22:48:02.791590 24042 manager.cpp:163] Terminating resource provider 
> 765391c2-9485-4614-a6e5-45e67ef7a92c
> ...
> ../../src/tests/storage_local_resource_provider_tests.cpp:1387: Failure
>   Expected: v1::OPERATION_FINISHED
>   Which is: OPERATION_FINISHED
> To be equal to: update->status().state()
>   Which is: OPERATION_ERROR
> ../../src/tests/storage_local_resource_provider_tests.cpp:1393: Failure
> Failed to wait 15secs for offers
> ...
> ../../src/tests/storage_local_resource_provider_tests.cpp:1361: Failure
> Actual function call count doesn't match EXPECT_CALL(*scheduler, offers(_, 
> v1::scheduler::OffersHaveAnyResource( std::bind(isStoragePool, lambda::_1, 
> "test2"...
>  Expected: to be called once
>Actual: never called - unsatisfied and active
> ```
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 69866: Updated SLRP test `ProfileDisappeared` to request operation feedbacks.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69866/
---

(Updated Jan. 30, 2019, 11:41 p.m.)


Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.


Changes
---

Made the definition of the `v1::CREATE_DISK` helper more consistent with other 
helpers.


Bugs: MESOS-9537
https://issues.apache.org/jira/browse/MESOS-9537


Repository: mesos


Description
---

This patch updates `StorageLocalResourceProviderTest.ProfileDisappeared`
to use the v1 scheduler API to request operation feedbacks, so
MESOS-9537 would be triggered when an outstanding `UPDATE_STATE` call
from the resource provider races with an offer operation.


Diffs (updated)
-

  src/tests/mesos.hpp c2a5e5531de7498241e537ef1699e1a5e669b550 
  src/tests/storage_local_resource_provider_tests.cpp 
e8ed20f818ed7f1a3ce15758ea3c366520443377 


Diff: https://reviews.apache.org/r/69866/diff/2/

Changes: https://reviews.apache.org/r/69866/diff/1-2/


Testing
---

`make check`

Ran the test ~1000 times.

This test will fail without r/69858:
```
E0130 22:48:02.790385 24040 provider.cpp:2903] Failed to update status of 
operation (uuid: 2bf6813c-bde9-4c0e-8831-a779f5dc50ac): Mismatched framework ID 
for operation status update OPERATION_DROPPED (Status UUID: 
8a625c40-96e8-46e2-8050-69843032969a) for operation UUID 
2bf6813c-bde9-4c0e-8831-a779f5dc50ac on agent 
bfa6e505-54b7-4bb8-a8aa-99fb12291fb1-S0 (expected 
bfa6e505-54b7-4bb8-a8aa-99fb12291fb1- got no framework ID)
I0130 22:48:02.791590 24042 manager.cpp:163] Terminating resource provider 
765391c2-9485-4614-a6e5-45e67ef7a92c
...
../../src/tests/storage_local_resource_provider_tests.cpp:1387: Failure
  Expected: v1::OPERATION_FINISHED
  Which is: OPERATION_FINISHED
To be equal to: update->status().state()
  Which is: OPERATION_ERROR
../../src/tests/storage_local_resource_provider_tests.cpp:1393: Failure
Failed to wait 15secs for offers
...
../../src/tests/storage_local_resource_provider_tests.cpp:1361: Failure
Actual function call count doesn't match EXPECT_CALL(*scheduler, offers(_, 
v1::scheduler::OffersHaveAnyResource( std::bind(isStoragePool, lambda::_1, 
"test2"...
 Expected: to be called once
   Actual: never called - unsatisfied and active
```


Thanks,

Chun-Hung Hsiao

Re: Review Request 69656: Fixed test `MesosContainerizerSlaveRecoveryTest.ResourceStatistics`.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69656/#review212468
---



Patch looks great!

Reviews applied: [69656]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Jan. 3, 2019, 5:16 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69656/
> ---
> 
> (Updated Jan. 3, 2019, 5:16 a.m.)
> 
> 
> Review request for mesos, Joseph Wu and Till Toenshoff.
> 
> 
> Bugs: MESOS-5048
> https://issues.apache.org/jira/browse/MESOS-5048
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> `MesosContainerizerSlaveRecoveryTest.ResourceStatistics` is flaky
> due to a race between executor shutdown (due to never getting any
> tasks) and the test querying resource statistics. If the executor
> is shutdown before the statistics query, the test will fail.
> 
> This patch fixes the test by explicitly waiting for the task to
> be delivered and task status transition to `TASK_RUNNING` before
> restarting the agent. This way, the executor will not be shutdown
> after agent restart. Hence there will be no race.
> 
> 
> Diffs
> -
> 
>   src/tests/slave_recovery_tests.cpp 0eb47e2bdf6a46fc21b59bb85b4b89181087ccd3 
> 
> 
> Diff: https://reviews.apache.org/r/69656/diff/1/
> 
> 
> Testing
> ---
> 
> ran `MesosContainerizerSlaveRecoveryTest.ResourceStatistics` continuously 
> without failure.
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Review Request 69866: Updated SLRP test `ProfileDisappeared` to request operation feedbacks.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69866/
---

Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.


Bugs: MESOS-9537
https://issues.apache.org/jira/browse/MESOS-9537


Repository: mesos


Description
---

This patch updates `StorageLocalResourceProviderTest.ProfileDisappeared`
to use the v1 scheduler API to request operation feedbacks, so
MESOS-9537 would be triggered when an outstanding `UPDATE_STATE` call
from the resource provider races with an offer operation.


Diffs
-

  src/tests/mesos.hpp c2a5e5531de7498241e537ef1699e1a5e669b550 
  src/tests/storage_local_resource_provider_tests.cpp 
e8ed20f818ed7f1a3ce15758ea3c366520443377 


Diff: https://reviews.apache.org/r/69866/diff/1/


Testing
---

`make check`

Ran the test ~1000 times.

This test will fail without r/69858.


Thanks,

Chun-Hung Hsiao

Re: Review Request 69858: Persisted intentionally dropped operations in SLRP.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69858/
---

(Updated Jan. 30, 2019, 10:35 p.m.)


Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.


Bugs: MESOS-9537
https://issues.apache.org/jira/browse/MESOS-9537


Repository: mesos


Description
---

If an operation is dropped intentionally (e.g., because of a resource
version mismatch), the operation should be persisted so no conflicting
status update would be generated for operation reconciliation.


Diffs
-

  src/resource_provider/storage/provider.cpp 
d6e20a549ede189c757ae3ae922ab7cb86d2be2c 
  src/resource_provider/storage/provider_process.hpp PRE-CREATION 


Diff: https://reviews.apache.org/r/69858/diff/3/


Testing (updated)
---

`make check`

Additional test MESOS-9537 is added later in chain.


Thanks,

Chun-Hung Hsiao

Re: Review Request 69858: Persisted intentionally dropped operations in SLRP.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69858/#review212467
---



PASS: Mesos patch 69858 was successfully built and tested.

Reviews applied: `['69858']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2835/mesos-review-69858

- Mesos Reviewbot Windows


On Jan. 30, 2019, 9:32 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69858/
> ---
> 
> (Updated Jan. 30, 2019, 9:32 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.
> 
> 
> Bugs: MESOS-9537
> https://issues.apache.org/jira/browse/MESOS-9537
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> If an operation is dropped intentionally (e.g., because of a resource
> version mismatch), the operation should be persisted so no conflicting
> status update would be generated for operation reconciliation.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> d6e20a549ede189c757ae3ae922ab7cb86d2be2c 
>   src/resource_provider/storage/provider_process.hpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69858/diff/3/
> 
> 
> Testing
> ---
> 
> Test will be added later in chain.
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 69863: Removed an unnecessary '.get()' from a master test.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69863/#review212466
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['69863']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2834/mesos-review-69863

Relevant logs:

- 
[mesos-tests.log](http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2834/mesos-review-69863/logs/mesos-tests.log):

```
W0130 21:42:23.961591 25700 slave.cpp:3934] Ignoring shutdown framework 
579e4b59-a671-4c18-8102-475d15ce84cc- because it is terminating
I0130 21:42:23.965603 22676 master.cpp:1269] Agent 
579e4b59-a671-4c18-8102-475d15ce84cc-S0 at slave(477)@192.10.1.6:55252 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net) disconnected
I0130 21:42:23.965603 22676 master.cpp:3272] Disconnecting agent 
579e4b59-a671-4c18-8102-475d15ce84cc-S0 at slave(477)@192.10.1.6:55252 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0130 21:42:23.966588 22676 master.cpp:3291] Deactivating agent 
579e4b59-a671-4c18-8102-475d15ce84cc-S0 at slave(477)@192.10.1.6:55252 
(windows-02.chtsmhjxogyevckjfayqqcnjda.xx.internal.cloudapp.net)
I0130 21:42:23.966588 26720 hierarchical.cpp:358] Removed framework 
579e4b59-a671-4c18-8102-475d15ce84cc-
I0130 21:42:23.966588 26720 hierarchical.cpp:793] Agent 
579e4b59-a671-4c18-8102-475d15ce84cc-S0 deactivated
I0130 21:42:23.967607 15788 containerizer.cpp:2477] Destroying container 
dbdeea83-333d-4256-8814-4139694e9cf0 in RUNNING state
I0130 21:42:23.967607 15788 containerizer.cpp:3144] Transitioning the state of 
container dbdeea83-333d-4256-8814-4139694e9cf0 from RUNNING to DESTROYING
I0130 21:42:23.968605 15788 launcher.cpp:161] Asked to destroy container 
dbdeea83-333d-4256-8814-4139694e9cf0
W0130 21:42:23.969594 27680 process.cpp:1423] Failed to recv on socket 
WindowsFD::Type::SOCKET=5716 to peer '192.10.1.6:57153': IO failed with error 
code: The specified network name is no longer available.

W0130 21:42:23.969594 27680 process.cpp:838] Failed to recv on socket 
WindowsFD::Type::SOCKET=6408 to peer '192.10.1.6:57154': IO failed with error 
code: The specified network name is no longer available.

I0130 21:42:24.056852 22676 containerizer.cpp:2983] Container 
dbdeea83-333d-4256-8814-413969[   OK ] 
IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (687 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (704 ms total)

[--] Global test environment tear-down
[==] 1095 tests from 104 test cases ran. (507512 ms total)
[  PASSED  ] 1094 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DockerFetcherPluginTest.INTERNET_CURL_FetchImage

 1 FAILED TEST
  YOU HAVE 231 DISABLED TESTS

4e9cf0 has exited
I0130 21:42:24.086869 25700 master.cpp:1109] Master terminating
I0130 21:42:24.087864 22676 hierarchical.cpp:644] Removed agent 
579e4b59-a671-4c18-8102-475d15ce84cc-S0
I0130 21:42:24.490849 27680 process.cpp:927] Stopped the socket accept loop
```

- Mesos Reviewbot Windows


On Jan. 30, 2019, 9:32 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69863/
> ---
> 
> (Updated Jan. 30, 2019, 9:32 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Removed an unnecessary '.get()' from a master test.
> 
> 
> Diffs
> -
> 
>   src/tests/master_tests.cpp 506d45af416f10787391f226ea62101732ee0850 
> 
> 
> Diff: https://reviews.apache.org/r/69863/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 69863: Removed an unnecessary '.get()' from a master test.

[Joseph Wu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69863/#review212465
---


Ship it!




Ship It!

- Joseph Wu


On Jan. 30, 2019, 12:32 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69863/
> ---
> 
> (Updated Jan. 30, 2019, 12:32 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Removed an unnecessary '.get()' from a master test.
> 
> 
> Diffs
> -
> 
>   src/tests/master_tests.cpp 506d45af416f10787391f226ea62101732ee0850 
> 
> 
> Diff: https://reviews.apache.org/r/69863/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 69863: Removed an unnecessary '.get()' from a master test.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69863/#review212463
---


Ship it!




Ship It!

- Benjamin Bannier


On Jan. 30, 2019, 9:32 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69863/
> ---
> 
> (Updated Jan. 30, 2019, 9:32 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Removed an unnecessary '.get()' from a master test.
> 
> 
> Diffs
> -
> 
>   src/tests/master_tests.cpp 506d45af416f10787391f226ea62101732ee0850 
> 
> 
> Diff: https://reviews.apache.org/r/69863/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 69858: Persisted intentionally dropped operations in SLRP.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69858/
---

(Updated Jan. 30, 2019, 8:32 p.m.)


Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.


Changes
---

Fixed a test issue.


Bugs: MESOS-9537
https://issues.apache.org/jira/browse/MESOS-9537


Repository: mesos


Description
---

If an operation is dropped intentionally (e.g., because of a resource
version mismatch), the operation should be persisted so no conflicting
status update would be generated for operation reconciliation.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
d6e20a549ede189c757ae3ae922ab7cb86d2be2c 
  src/resource_provider/storage/provider_process.hpp PRE-CREATION 


Diff: https://reviews.apache.org/r/69858/diff/3/

Changes: https://reviews.apache.org/r/69858/diff/2-3/


Testing
---

Test will be added later in chain.


Thanks,

Chun-Hung Hsiao

Review Request 69863: Removed an unnecessary '.get()' from a master test.

[Greg Mann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69863/
---

Review request for mesos, Benjamin Bannier and Joseph Wu.


Repository: mesos


Description
---

Removed an unnecessary '.get()' from a master test.


Diffs
-

  src/tests/master_tests.cpp 506d45af416f10787391f226ea62101732ee0850 


Diff: https://reviews.apache.org/r/69863/diff/1/


Testing
---


Thanks,

Greg Mann

Re: Review Request 69855: Added an example of Seccomp profile.

[Gilbert Song]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69855/#review212459
---


Ship it!




Ship It!

- Gilbert Song


On Jan. 29, 2019, 10:32 a.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69855/
> ---
> 
> (Updated Jan. 29, 2019, 10:32 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, Jie Yu, and Qian Zhang.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> See summary.
> 
> 
> Diffs
> -
> 
>   docs/examples/seccomp_default.json PRE-CREATION 
>   docs/isolators/linux-seccomp.md fa11feaeada2bcc6b9c782fa0e2ec8adad70e10f 
> 
> 
> Diff: https://reviews.apache.org/r/69855/diff/1/
> 
> 
> Testing
> ---
> 
> None: not a functional change.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 69820: Added containment check for `ResourceQuantities`.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69820/#review212458
---



Patch looks great!

Reviews applied: [69861, 69818, 69819, 69820]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Jan. 29, 2019, 4:20 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69820/
> ---
> 
> (Updated Jan. 29, 2019, 4:20 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9523
> https://issues.apache.org/jira/browse/MESOS-9523
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds `contains` function for `ResourceQuantities`. The
> function has the same semantics as `Resources::contains`.
> 
> 
> Diffs
> -
> 
>   src/common/resource_quantities.hpp 11eb426104577b7977c2307df3e4917085cd 
>   src/common/resource_quantities.cpp 320983929cd7d14973c4b98d6ed5338de690ff5f 
>   src/tests/resource_quantities_tests.cpp 
> 435a4949b95e9a83be73781388eb4be9c7da695b 
> 
> 
> Diff: https://reviews.apache.org/r/69820/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69820: Added containment check for `ResourceQuantities`.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69820/#review212457
---



PASS: Mesos patch 69820 was successfully built and tested.

Reviews applied: `['69861', '69818', '69819', '69820']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2832/mesos-review-69820

- Mesos Reviewbot Windows


On Jan. 29, 2019, 4:20 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69820/
> ---
> 
> (Updated Jan. 29, 2019, 4:20 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9523
> https://issues.apache.org/jira/browse/MESOS-9523
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds `contains` function for `ResourceQuantities`. The
> function has the same semantics as `Resources::contains`.
> 
> 
> Diffs
> -
> 
>   src/common/resource_quantities.hpp 11eb426104577b7977c2307df3e4917085cd 
>   src/common/resource_quantities.cpp 320983929cd7d14973c4b98d6ed5338de690ff5f 
>   src/tests/resource_quantities_tests.cpp 
> 435a4949b95e9a83be73781388eb4be9c7da695b 
> 
> 
> Diff: https://reviews.apache.org/r/69820/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69821: Enforced minimal allocatable resources in the hierarchical allocator.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69821/#review212456
---



PASS: Mesos patch 69821 was successfully built and tested.

Reviews applied: `['69861', '69818', '69862', '69821']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2833/mesos-review-69821

- Mesos Reviewbot Windows


On Jan. 30, 2019, 4:08 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69821/
> ---
> 
> (Updated Jan. 30, 2019, 4:08 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9523
> https://issues.apache.org/jira/browse/MESOS-9523
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch modifies the hierarchical allocator to take
> framework-specified minimal allocatable resources into account.
> 
> While previously the allocator was inspecting the minimal allocatable
> resources specified in its options, it now inspects a dynamically
> calculated set of minimal allocatable resources. This set is populated
> with any minimal allocatable resources specified in the allocator's
> options and any framework's minimal allocatable resources. For a
> resource to be allocatable it needs to contain any of these minimal
> requirements.
> 
> If a framework does not specify minimal allocatable resource
> requirements, its minimal requirements are set to the globally
> configured option.
> 
> We also adjust the allocator to take a framework's minimal resource
> requirements into account when checking whether a resource can be
> allocated to a particular framework. The check is performed at the same
> time we check whether a framework filtered a particular resource. This
> avoids offering resources to frameworks which the framework would never
> have considered allocatable itself (e.g., given the global option of the
> allocator).
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
>   src/tests/hierarchical_allocator_tests.cpp 
> cc88afbad1b4e6bf707cb13b50c964aa01f9a3ee 
> 
> 
> Diff: https://reviews.apache.org/r/69821/diff/3/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69861: Declared `HierarchicalAllocatorProcess::allocatable` `const`.

[Benjamin Bannier]

> On Jan. 30, 2019, 5:09 p.m., Benjamin Mahler wrote:
> > Thanks! Probably don't need the duplicate description in the commit message?
> 
> Benjamin Bannier wrote:
> The way we have our reviewboard interaction set up, not manually 
> adjusting a redundant description is actually the easier way: During 
> post-review it gets populated automatically with the summary, and 
> apply-review removes the redundant information again. 
> https://github.com/apache/mesos/commit/69c918a72aa7e15ce12f8afc12203a115c4cd634
> 
> Benjamin Mahler wrote:
> Is there a way to prevent the redundancy at post-reviews time?

IIRC there were some difficulties with us wanting to populate some fields, so 
this would have required injecting this between `rbt` and the actual post. A 
benefit of doing this at apply-review time is that it can deal even with 
manually injected, redundant descriptions.


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69861/#review212452
---


On Jan. 30, 2019, 4:03 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69861/
> ---
> 
> (Updated Jan. 30, 2019, 4:03 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Declared `HierarchicalAllocatorProcess::allocatable` `const`.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
> 
> 
> Diff: https://reviews.apache.org/r/69861/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69861: Declared `HierarchicalAllocatorProcess::allocatable` `const`.

[Benjamin Mahler]

> On Jan. 30, 2019, 4:09 p.m., Benjamin Mahler wrote:
> > Thanks! Probably don't need the duplicate description in the commit message?
> 
> Benjamin Bannier wrote:
> The way we have our reviewboard interaction set up, not manually 
> adjusting a redundant description is actually the easier way: During 
> post-review it gets populated automatically with the summary, and 
> apply-review removes the redundant information again. 
> https://github.com/apache/mesos/commit/69c918a72aa7e15ce12f8afc12203a115c4cd634

Is there a way to prevent the redundancy at post-reviews time?


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69861/#review212452
---


On Jan. 30, 2019, 3:03 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69861/
> ---
> 
> (Updated Jan. 30, 2019, 3:03 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Declared `HierarchicalAllocatorProcess::allocatable` `const`.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
> 
> 
> Diff: https://reviews.apache.org/r/69861/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69861: Declared `HierarchicalAllocatorProcess::allocatable` `const`.

[Benjamin Bannier]

> On Jan. 30, 2019, 5:09 p.m., Benjamin Mahler wrote:
> > Thanks! Probably don't need the duplicate description in the commit message?

The way we have our reviewboard interaction set up, not manually adjusting a 
redundant description is actually the easier way: During post-review it gets 
populated automatically with the summary, and apply-review removes the 
redundant information again. 
https://github.com/apache/mesos/commit/69c918a72aa7e15ce12f8afc12203a115c4cd634


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69861/#review212452
---


On Jan. 30, 2019, 4:03 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69861/
> ---
> 
> (Updated Jan. 30, 2019, 4:03 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Declared `HierarchicalAllocatorProcess::allocatable` `const`.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
> 
> 
> Diff: https://reviews.apache.org/r/69861/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69861: Declared `HierarchicalAllocatorProcess::allocatable` `const`.

[Benjamin Mahler]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69861/#review212452
---


Ship it!




Thanks! Probably don't need the duplicate description in the commit message?

- Benjamin Mahler


On Jan. 30, 2019, 3:03 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69861/
> ---
> 
> (Updated Jan. 30, 2019, 3:03 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Declared `HierarchicalAllocatorProcess::allocatable` `const`.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
> 
> 
> Diff: https://reviews.apache.org/r/69861/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69821: Enforced minimal allocatable resources in the hierarchical allocator.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69821/
---

(Updated Jan. 30, 2019, 4:08 p.m.)


Review request for mesos, Benjamin Mahler and Meng Zhu.


Changes
---

WIP.


Bugs: MESOS-9523
https://issues.apache.org/jira/browse/MESOS-9523


Repository: mesos


Description
---

This patch modifies the hierarchical allocator to take
framework-specified minimal allocatable resources into account.

While previously the allocator was inspecting the minimal allocatable
resources specified in its options, it now inspects a dynamically
calculated set of minimal allocatable resources. This set is populated
with any minimal allocatable resources specified in the allocator's
options and any framework's minimal allocatable resources. For a
resource to be allocatable it needs to contain any of these minimal
requirements.

If a framework does not specify minimal allocatable resource
requirements, its minimal requirements are set to the globally
configured option.

We also adjust the allocator to take a framework's minimal resource
requirements into account when checking whether a resource can be
allocated to a particular framework. The check is performed at the same
time we check whether a framework filtered a particular resource. This
avoids offering resources to frameworks which the framework would never
have considered allocatable itself (e.g., given the global option of the
allocator).


Diffs (updated)
-

  src/master/allocator/mesos/hierarchical.hpp 
ca1638390d89e2a81efd9d6d4a28b863c79723c4 
  src/master/allocator/mesos/hierarchical.cpp 
f1f3894058a8e3f008013cb269744bd36c0e31b3 
  src/tests/hierarchical_allocator_tests.cpp 
cc88afbad1b4e6bf707cb13b50c964aa01f9a3ee 


Diff: https://reviews.apache.org/r/69821/diff/3/

Changes: https://reviews.apache.org/r/69821/diff/2-3/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Review Request 69862: Validated static framework offer filters.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69862/
---

Review request for mesos, Benjamin Mahler and Meng Zhu.


Bugs: MESOS-9523
https://issues.apache.org/jira/browse/MESOS-9523


Repository: mesos


Description
---

Validated static framework offer filters.


Diffs
-

  src/common/validation.hpp 5929ac24fa7d7d2f7f08bc30c5071716e6cf46cb 
  src/common/validation.cpp 4a64fc8ac83fd741eb2cbc70772322b347dbcb4e 
  src/master/validation.hpp a5bdff6a9301631dac9970568a1346460939c861 
  src/master/validation.cpp a71edeb7827b534e27ad3e3398abe555bf36e741 


Diff: https://reviews.apache.org/r/69862/diff/1/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Re: Review Request 69818: Added offer filters to static framework configuration.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69818/
---

(Updated Jan. 30, 2019, 4:04 p.m.)


Review request for mesos, Benjamin Mahler and Meng Zhu.


Changes
---

WIP.


Summary (updated)
-

Added offer filters to static framework configuration.


Bugs: MESOS-9523
https://issues.apache.org/jira/browse/MESOS-9523


Repository: mesos


Description (updated)
---

This patch adds a new field to `FrameworkInfo` which can be used by
frameworks to configure static, per-role resource requirements. We
currently support specifying minimal allocatable resources which can
e.g., be interpreted by allocators to filter out resources not relevant
to frameworks, or make allocations below globally configured minimal
allocatable resources for particular frameworks.


Diffs (updated)
-

  include/mesos/mesos.proto 3477b7db4dc64187aa55e8e4feb66a62f7951429 
  include/mesos/v1/mesos.proto 692940bc862e61b9899aa159f29b662f3aade033 
  src/common/resource_quantities.hpp 11eb426104577b7977c2307df3e4917085cd 
  src/common/resource_quantities.cpp 320983929cd7d14973c4b98d6ed5338de690ff5f 


Diff: https://reviews.apache.org/r/69818/diff/4/

Changes: https://reviews.apache.org/r/69818/diff/3-4/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Review Request 69861: Declared `HierarchicalAllocatorProcess::allocatable` `const`.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69861/
---

Review request for mesos, Benjamin Mahler and Meng Zhu.


Repository: mesos


Description
---

Declared `HierarchicalAllocatorProcess::allocatable` `const`.


Diffs
-

  src/master/allocator/mesos/hierarchical.hpp 
ca1638390d89e2a81efd9d6d4a28b863c79723c4 
  src/master/allocator/mesos/hierarchical.cpp 
f1f3894058a8e3f008013cb269744bd36c0e31b3 


Diff: https://reviews.apache.org/r/69861/diff/1/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Re: Review Request 69821: Enforced minimal allocatable resources in the hierarchical allocator.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69821/#review212450
---



Patch looks great!

Reviews applied: [69818, 69819, 69820, 69821]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Jan. 29, 2019, 4:20 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69821/
> ---
> 
> (Updated Jan. 29, 2019, 4:20 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-9523
> https://issues.apache.org/jira/browse/MESOS-9523
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch modifies the hierarchical allocator to take
> framework-specified minimal allocatable resources into account.
> 
> While previously the allocator was inspecting the minimal allocatable
> resources specified in its options, it now inspects a dynamically
> calculated set of minimal allocatable resources. This set is populated
> with any minimal allocatable resources specified in the allocator's
> options and any framework's minimal allocatable resources. For a
> resource to be allocatable it needs to contain any of these minimal
> requirements.
> 
> If a framework does not specify minimal allocatable resource
> requirements, its minimal requirements are set to the globally
> configured option.
> 
> We also adjust the allocator to take a framework's minimal resource
> requirements into account when checking whether a resource can be
> allocated to a particular framework. The check is performed at the same
> time we check whether a framework filtered a particular resource. This
> avoids offering resources to frameworks which the framework would never
> have considered allocatable itself (e.g., given the global option of the
> allocator).
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> ca1638390d89e2a81efd9d6d4a28b863c79723c4 
>   src/master/allocator/mesos/hierarchical.cpp 
> f1f3894058a8e3f008013cb269744bd36c0e31b3 
>   src/tests/hierarchical_allocator_tests.cpp 
> cc88afbad1b4e6bf707cb13b50c964aa01f9a3ee 
> 
> 
> Diff: https://reviews.apache.org/r/69821/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 69858: Persisted intentionally dropped operations in SLRP.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69858/#review212448
---



PASS: Mesos patch 69858 was successfully built and tested.

Reviews applied: `['69858']`

All the build artifacts available at: 
http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2831/mesos-review-69858

- Mesos Reviewbot Windows


On Jan. 30, 2019, 5:38 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69858/
> ---
> 
> (Updated Jan. 30, 2019, 5:38 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gastón Kleiman, and Greg Mann.
> 
> 
> Bugs: MESOS-9537
> https://issues.apache.org/jira/browse/MESOS-9537
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> If an operation is dropped intentionally (e.g., because of a resource
> version mismatch), the operation should be persisted so no conflicting
> status update would be generated for operation reconciliation.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> d6e20a549ede189c757ae3ae922ab7cb86d2be2c 
>   src/resource_provider/storage/provider_process.hpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69858/diff/2/
> 
> 
> Testing
> ---
> 
> Test will be added later in chain.
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 69860: Disabled parallel test execution for reviewbot.

[Alexander Rukletsov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69860/#review212442
---


Ship it!




Ship It!

- Alexander Rukletsov


On Jan. 30, 2019, 8:09 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69860/
> ---
> 
> (Updated Jan. 30, 2019, 8:09 a.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> While we globally disabled parallel test execution for reviewbot jobs
> in `37532fae01d`, we missed that the configuration was overwritten
> downstream.
> 
> This patch disables parallel tests also in the script ultimately
> performing the final configuration.
> 
> 
> Diffs
> -
> 
>   support/verify-reviews.py 71326d34bb649e27a3a2901867d31a2a1fffd4e9 
> 
> 
> Diff: https://reviews.apache.org/r/69860/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Review Request 69860: Disabled parallel test execution for reviewbot.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69860/
---

Review request for mesos, Alexander Rukletsov and Benjamin Mahler.


Repository: mesos


Description
---

While we globally disabled parallel test execution for reviewbot jobs
in `37532fae01d`, we missed that the configuration was overwritten
downstream.

This patch disables parallel tests also in the script ultimately
performing the final configuration.


Diffs
-

  support/verify-reviews.py 71326d34bb649e27a3a2901867d31a2a1fffd4e9 


Diff: https://reviews.apache.org/r/69860/diff/1/


Testing
---


Thanks,

Benjamin Bannier