Re: Review Request 64466: Added a test to check shared resources accounting in quota enforcement.

2017-12-15 Thread Benjamin Mahler

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64466/#review194005
---


Ship it!




Meng and I went over this patch together and made a few modifications.

- Benjamin Mahler


On Dec. 9, 2017, 12:48 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64466/
> ---
> 
> (Updated Dec. 9, 2017, 12:48 a.m.)
> 
> 
> Review request for mesos, Anindya Sinha, Benjamin Mahler, Michael Park, and 
> Jiang Yan Xu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test verifies that when enforcing quota limit, shared resources
> that are part of a role's reserved-allocated resources are only
> charged once even when they are offered multiple times.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 862f4683da04d37d9fe9f471d6ec9cd7751f39ec 
> 
> 
> Diff: https://reviews.apache.org/r/64466/diff/4/
> 
> 
> Testing
> ---
> 
> make check.
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>



Re: Review Request 64465: Added a test for non-quota role reservation allocation.

2017-12-15 Thread Benjamin Mahler

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64465/#review194003
---


Fix it, then Ship it!





src/tests/hierarchical_allocator_tests.cpp
Lines 1466-1497 (patched)


Can we combine these into a single path?


- Benjamin Mahler


On Dec. 8, 2017, 10:16 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64465/
> ---
> 
> (Updated Dec. 8, 2017, 10:16 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8293
> https://issues.apache.org/jira/browse/MESOS-8293
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test checks against the symptom of failing to allocate
> non-quota role's reservations if resources are set aside for
> the quota limit headroom. See MESOS-8293.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> f5fb47ed09682ebdd047aec7e79a86597ee09f53 
> 
> 
> Diff: https://reviews.apache.org/r/64465/diff/5/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>



Re: Review Request 64636: Fixed a bug with quota headroom that can leave reservations unallocated.

2017-12-15 Thread Benjamin Mahler

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64636/#review193993
---


Fix it, then Ship it!




Perhaps the required / available headroom computations should be contained in 
functions?

```
Resources requiredQuotaHeadroom() const;
Resources availableQuotaHeadroom() const;
```

I guess it's hard to then define how to grow/shrink the headroom since the 
definition of what consists of headroom is spread between the function and the 
allocation loop that's trying to increment it instead of re-compute it?


src/master/allocator/mesos/hierarchical.cpp
Lines 1826 (patched)


s/name/role/



src/master/allocator/mesos/hierarchical.cpp
Lines 1833 (patched)


Perhaps an = here?



src/master/allocator/mesos/hierarchical.cpp
Lines 1852-1854 (original), 1878-1880 (patched)


What's this for?



src/master/allocator/mesos/hierarchical.cpp
Lines 2020-2021 (patched)


This seemed a little backwards when I read it, I was expecting to see 
something more like:

available headroom - what we're going to allocate >= required headroom

So:

```
(availableHeadroom - headroomToAllocate).contains(requiredHeadroom)
```



src/master/allocator/mesos/hierarchical.cpp
Lines 2032 (patched)


This should be unstripped? Looks like we need to strip for the headroom 
calculation but not here?


- Benjamin Mahler


On Dec. 15, 2017, 2:34 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64636/
> ---
> 
> (Updated Dec. 15, 2017, 2:34 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-8293
> https://issues.apache.org/jira/browse/MESOS-8293
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Also fixed a bug where quota headroom may wrongly include revocable
> resources.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> fccd71c96fe8e4d914e19b5ea8d8f69e9ebf2406 
> 
> 
> Diff: https://reviews.apache.org/r/64636/diff/2/
> 
> 
> Testing
> ---
> 
> make check.
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>



Re: Review Request 64436: Toggled the authenticate_http_readwrite flag in MesosTest.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64436/#review194004
---



PASS: Mesos patch 64436 was successfully built and tested.

Reviews applied: `['64436']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64436

- Mesos Reviewbot Windows


On Dec. 16, 2017, 12:45 a.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64436/
> ---
> 
> (Updated Dec. 16, 2017, 12:45 a.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The default executor authentication mechanism depends on SSL support
> being enabled in the build. Multiple tests that invoke the default
> executor have compilation time checks for SSL support so they
> can disable the Mesos agent authenticate_http_readwrite flag in
> this case.  Rather than duplicating this check to multiple tests,
> set it to an appropriate default in the MesosTest fixture.
> 
> 
> Diffs
> -
> 
>   src/tests/api_tests.cpp 86cbba4fab5e7a45298d17f3f2969391cc18be68 
>   src/tests/check_tests.cpp 2a71f41bad1a7eea2763addf5fc35d648bd46cc6 
>   src/tests/container_daemon_tests.cpp 
> c6152571d503d7401c0f2a7099d020e3d3db26a0 
>   src/tests/containerizer/environment_secret_isolator_tests.cpp 
> fd1cd468bd1e4c2455bb3315b085d38b2543bc6f 
>   src/tests/containerizer/linux_capabilities_isolator_tests.cpp 
> d3c456826db2c1844ebfd55acbd766d6fb7eaede 
>   src/tests/containerizer/nvidia_gpu_isolator_tests.cpp 
> 99431784dac41da531b92c9a0e3c4772a6bb57ae 
>   src/tests/containerizer/ports_isolator_tests.cpp 
> 343803028afce01874a054879f5ec39f2b85286a 
>   src/tests/containerizer/posix_rlimits_isolator_tests.cpp 
> f639cac900f8ac5d0bc2eb856e90cb4881d85ec3 
>   src/tests/containerizer/volume_host_path_isolator_tests.cpp 
> e4d0734ffdb8e9d1c7dcc2cb5aa75bbb014b6415 
>   src/tests/default_executor_tests.cpp 
> ee6dfbde02277caf330b72f6e8210924150a428c 
>   src/tests/health_check_tests.cpp bb909c5ea4a51b562ac753083fda002195ac08ad 
>   src/tests/master_slave_reconciliation_tests.cpp 
> 1a9b40a9a269313fc6c858ab5b2a9901c5e12352 
>   src/tests/master_tests.cpp e41bd942b1ab55130672f5d5b7044b376a21a7f1 
>   src/tests/mesos.cpp 103128308e64ec1e1a5f2f979a42b96fbc0cd8d6 
>   src/tests/scheduler_tests.cpp ecfce4d112464a76c14603e40c20585ec25e0ca7 
> 
> 
> Diff: https://reviews.apache.org/r/64436/diff/2/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>



Re: Review Request 64658: Modified SLRP to use the VolumeProfileAdaptor module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64658/
---

(Updated Dec. 16, 2017, 1:34 a.m.)


Review request for mesos, Jie Yu and Joseph Wu.


Changes
---

Added a `RECONCILING` state for profile updates.


Repository: mesos


Description
---

This changes the Storage Local Resource Provider's source of profile
information from only using the default to using a VolumeProfileAdaptor
module.

This patch is based on https://reviews.apache.org/r/64616.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
158b6b408002209aa9a79a6772da30c984aad61a 


Diff: https://reviews.apache.org/r/64658/diff/4/

Changes: https://reviews.apache.org/r/64658/diff/3-4/


Testing
---

See later in chain. This patch will fail SLRP tests. They are fixed in a 
followup patch.


Thanks,

Chun-Hung Hsiao



Re: Review Request 64659: Fixed unit tests for volume profile module integration.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64659/#review194002
---



FAIL: Failed to apply the dependent review: 64658.

Failed command: `python.exe .\support\apply-reviews.py -n -r 64658`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64659

Relevant logs:

- 
[apply-review-64658-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64659/logs/apply-review-64658-stdout.log):

```
error: patch failed: src/resource_provider/storage/provider.cpp:344
error: src/resource_provider/storage/provider.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Dec. 15, 2017, 3:56 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64659/
> ---
> 
> (Updated Dec. 15, 2017, 3:56 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Fixed unit tests for volume profile module integration.
> 
> 
> Diffs
> -
> 
>   src/tests/agent_resource_provider_config_api_tests.cpp 
> 3c68b732a1d41561f98206a55ac09d91229f330a 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 1f11825001768864b61fdebdb24168790d7ec4fe 
> 
> 
> Diff: https://reviews.apache.org/r/64659/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 64436: Toggled the authenticate_http_readwrite flag in MesosTest.

2017-12-15 Thread James Peach

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64436/
---

(Updated Dec. 16, 2017, 12:45 a.m.)


Review request for mesos and Greg Mann.


Changes
---

Rebased.


Repository: mesos


Description
---

The default executor authentication mechanism depends on SSL support
being enabled in the build. Multiple tests that invoke the default
executor have compilation time checks for SSL support so they
can disable the Mesos agent authenticate_http_readwrite flag in
this case.  Rather than duplicating this check to multiple tests,
set it to an appropriate default in the MesosTest fixture.


Diffs (updated)
-

  src/tests/api_tests.cpp 86cbba4fab5e7a45298d17f3f2969391cc18be68 
  src/tests/check_tests.cpp 2a71f41bad1a7eea2763addf5fc35d648bd46cc6 
  src/tests/container_daemon_tests.cpp c6152571d503d7401c0f2a7099d020e3d3db26a0 
  src/tests/containerizer/environment_secret_isolator_tests.cpp 
fd1cd468bd1e4c2455bb3315b085d38b2543bc6f 
  src/tests/containerizer/linux_capabilities_isolator_tests.cpp 
d3c456826db2c1844ebfd55acbd766d6fb7eaede 
  src/tests/containerizer/nvidia_gpu_isolator_tests.cpp 
99431784dac41da531b92c9a0e3c4772a6bb57ae 
  src/tests/containerizer/ports_isolator_tests.cpp 
343803028afce01874a054879f5ec39f2b85286a 
  src/tests/containerizer/posix_rlimits_isolator_tests.cpp 
f639cac900f8ac5d0bc2eb856e90cb4881d85ec3 
  src/tests/containerizer/volume_host_path_isolator_tests.cpp 
e4d0734ffdb8e9d1c7dcc2cb5aa75bbb014b6415 
  src/tests/default_executor_tests.cpp ee6dfbde02277caf330b72f6e8210924150a428c 
  src/tests/health_check_tests.cpp bb909c5ea4a51b562ac753083fda002195ac08ad 
  src/tests/master_slave_reconciliation_tests.cpp 
1a9b40a9a269313fc6c858ab5b2a9901c5e12352 
  src/tests/master_tests.cpp e41bd942b1ab55130672f5d5b7044b376a21a7f1 
  src/tests/mesos.cpp 103128308e64ec1e1a5f2f979a42b96fbc0cd8d6 
  src/tests/scheduler_tests.cpp ecfce4d112464a76c14603e40c20585ec25e0ca7 


Diff: https://reviews.apache.org/r/64436/diff/2/

Changes: https://reviews.apache.org/r/64436/diff/1-2/


Testing
---

make check (Fedora 27)


Thanks,

James Peach



Re: Review Request 64522: Made OfferOperationStatusUpdateManager tests pass on Windows.

2017-12-15 Thread Andrew Schwartzmeyer

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/#review194000
---




src/status_update_manager/status_update_manager_process.hpp
Lines 635-637 (patched)


This weird Windows-ism is biting us in a lot of places. We've had to fix it 
elsewhere previously, and I have a dirty tree with the same fix applied to 
`net::download`.

We should commit this as-is for now. I will follow up with a patch to 
`os::open` itself to default to this behavior on Windows, possibly with an 
optional argument to turn it off if needed. I have a hunch, however, that 
nothing in Mesos expects or wants the "text mode" of Windows where it 
translates all your newlines to CRLFs.


- Andrew Schwartzmeyer


On Dec. 15, 2017, 1:25 p.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64522/
> ---
> 
> (Updated Dec. 15, 2017, 1:25 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Greg Mann, and Jie Yu.
> 
> 
> Bugs: MESOS-8318
> https://issues.apache.org/jira/browse/MESOS-8318
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> See the summary.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> e575d8c31bb6bd63fc26f01e2deec5f44821e01c 
>   src/tests/offer_operation_status_update_manager_tests.cpp 
> cb71bc10f9a687f12ee4cca4f8fa8614f5ea2d58 
> 
> 
> Diff: https://reviews.apache.org/r/64522/diff/10/
> 
> 
> Testing
> ---
> 
> It took a while, but the Windows reviewbot is now happy =).
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>



Re: Review Request 64522: Made OfferOperationStatusUpdateManager tests pass on Windows.

2017-12-15 Thread Andrew Schwartzmeyer

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/#review193999
---


Ship it!




Ship It!

- Andrew Schwartzmeyer


On Dec. 15, 2017, 1:25 p.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64522/
> ---
> 
> (Updated Dec. 15, 2017, 1:25 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> See the summary.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> e575d8c31bb6bd63fc26f01e2deec5f44821e01c 
>   src/tests/offer_operation_status_update_manager_tests.cpp 
> cb71bc10f9a687f12ee4cca4f8fa8614f5ea2d58 
> 
> 
> Diff: https://reviews.apache.org/r/64522/diff/10/
> 
> 
> Testing
> ---
> 
> It took a while, but the Windows reviewbot is now happy =).
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>



Re: Review Request 64659: Fixed unit tests for volume profile module integration.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64659/
---

(Updated Dec. 15, 2017, 11:56 p.m.)


Review request for mesos, Jie Yu and Joseph Wu.


Changes
---

Rebased on Joseph's new branch.


Repository: mesos


Description
---

Fixed unit tests for volume profile module integration.


Diffs (updated)
-

  src/tests/agent_resource_provider_config_api_tests.cpp 
3c68b732a1d41561f98206a55ac09d91229f330a 
  src/tests/storage_local_resource_provider_tests.cpp 
1f11825001768864b61fdebdb24168790d7ec4fe 


Diff: https://reviews.apache.org/r/64659/diff/2/

Changes: https://reviews.apache.org/r/64659/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/#review193998
---




src/resource_provider/uri_volume_profile.hpp
Lines 229 (patched)


Let's model this as a proto3 message. It'll be much easier to parse


- Jie Yu


On Dec. 15, 2017, 10:49 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64353/
> ---
> 
> (Updated Dec. 15, 2017, 10:49 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This example module shows how a VolumeProfile module might be
> implemented (and is a viable module in its own right).  The module
> can be configured to fetch a map of profiles from a URI (`file://` or
> `http(s)://`) and possibly cache this item for some time.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
>   src/resource_provider/uri_volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64353/diff/6/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64612: Added flag to pass VolumeProfileAdaptor module to SLRP.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64612/#review193997
---




src/resource_provider/local.cpp
Lines 39 (patched)


It's weird that LocalResourceProvider needs to be aware 
`volumeProfileAdaptor` which is very StorageLocalResourceProvider specific.

It's more appropriate to instantiate that in StorageLocalResourceProvider 
and make it a singleton.

probably passing the agent flags all the way to this method (or use 
`Parameters` to pass in provider specifc parameters).


- Jie Yu


On Dec. 14, 2017, 1:15 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64612/
> ---
> 
> (Updated Dec. 14, 2017, 1:15 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This adds an agent flag to choose the VolumeProfileAdaptor module
> to load and passes this information down to the storage resource
> providers. A single module can potentially be used by multiple
> storage resource providers.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/daemon.hpp a6d0013fa3645fd2b705351a86679f7fba13e7e3 
>   src/resource_provider/daemon.cpp f160a87c37d975d632db05694ca7b59f1d275821 
>   src/resource_provider/local.hpp 20bcc78d3fe847e03526fa59116bdbac92ec1e29 
>   src/resource_provider/local.cpp ae23c2000b7941aad92d3b83e4e905bdf042ebe8 
>   src/resource_provider/storage/provider.hpp 
> 5a371b19289c6e39fedd4cda65fa8be432d095e6 
>   src/resource_provider/storage/provider.cpp 
> f00fe58e12dcaa911ab3d05c3d7719c65a1484fb 
>   src/slave/flags.hpp 5918484825c3c813883c42465f3e5a5693d6a2cd 
>   src/slave/flags.cpp d4b95b7542733b22299bc19235cf99220534c022 
> 
> 
> Diff: https://reviews.apache.org/r/64612/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> Basically just a plumbing change to pass the module into the SLRP.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/#review193996
---




src/resource_provider/uri_volume_profile.hpp
Lines 106 (patched)


`!value.absolute()`


- Chun-Hung Hsiao


On Dec. 15, 2017, 10:49 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64353/
> ---
> 
> (Updated Dec. 15, 2017, 10:49 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This example module shows how a VolumeProfile module might be
> implemented (and is a viable module in its own right).  The module
> can be configured to fetch a map of profiles from a URI (`file://` or
> `http(s)://`) and possibly cache this item for some time.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
>   src/resource_provider/uri_volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64353/diff/6/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64354: Added tests for UriVolumeProfile module.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64354/#review193995
---


Ship it!




Ship It!

- Jie Yu


On Dec. 15, 2017, 7:29 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64354/
> ---
> 
> (Updated Dec. 15, 2017, 7:29 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> These tests mostly exercise the parsing logic used by the module
> and some of the interaction expected from callers of the module
> (i.e. the Storage Local Resource Provider).
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/tests/volume_profile_tests.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64354/diff/3/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/#review193994
---




src/resource_provider/uri_volume_profile.cpp
Lines 392-395 (patched)


Can you use proto3's default json->protobuf parsing:

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.util.json_util#JsonStringToMessage.details



src/resource_provider/uri_volume_profile.cpp
Lines 311 (patched)


Looks like a bunch of these can be simplified if we internally define a 
protobuf (proto3) for the content in the uri.

The parsing can just be proto3 json->protobuf parsing

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.util.json_util#JsonStringToMessage.details


- Jie Yu


On Dec. 15, 2017, 10:49 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64353/
> ---
> 
> (Updated Dec. 15, 2017, 10:49 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This example module shows how a VolumeProfile module might be
> implemented (and is a viable module in its own right).  The module
> can be configured to fetch a map of profiles from a URI (`file://` or
> `http(s)://`) and possibly cache this item for some time.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
>   src/resource_provider/uri_volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64353/diff/6/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Joseph Wu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/
---

(Updated Dec. 15, 2017, 2:49 p.m.)


Review request for mesos, Chun-Hung Hsiao and Jie Yu.


Changes
---

Fixed `--uri` flag validation lambda.


Bugs: MESOS-8251
https://issues.apache.org/jira/browse/MESOS-8251


Repository: mesos


Description
---

This example module shows how a VolumeProfile module might be
implemented (and is a viable module in its own right).  The module
can be configured to fetch a map of profiles from a URI (`file://` or
`http(s)://`) and possibly cache this item for some time.


Diffs (updated)
-

  src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
  src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
  src/resource_provider/uri_volume_profile.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/64353/diff/6/

Changes: https://reviews.apache.org/r/64353/diff/5-6/


Testing
---

See end of chain.


Thanks,

Joseph Wu



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/#review193991
---




src/resource_provider/uri_volume_profile.hpp
Lines 105 (patched)


We will always return an error if the value is an absolute path.


- Chun-Hung Hsiao


On Dec. 15, 2017, 7:29 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64353/
> ---
> 
> (Updated Dec. 15, 2017, 7:29 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This example module shows how a VolumeProfile module might be
> implemented (and is a viable module in its own right).  The module
> can be configured to fetch a map of profiles from a URI (`file://` or
> `http(s)://`) and possibly cache this item for some time.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
>   src/resource_provider/uri_volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64353/diff/5/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64352: Added default VolumeProfile module implementation.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64352/#review193989
---


Ship it!




Ship It!

- Jie Yu


On Dec. 15, 2017, 7:28 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64352/
> ---
> 
> (Updated Dec. 15, 2017, 7:28 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> By default, Mesos and the Storage Local Resource Provider (SLRP) will
> not expect profiles to be used. This is partly due to the experimental
> nature of the "profile" field and partly because some explicit operator
> intervention is required to get profiles defined. There is no default
> profile that is accepted by CSI plugins.
> 
> 
> Diffs
> -
> 
>   include/mesos/module/volume_profile.hpp PRE-CREATION 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/module/manager.cpp d93648ab3f95e17d28056e9f5eb75ebff30775e1 
>   src/resource_provider/volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64352/diff/3/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 64657: Added tests for VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64657/#review193987
---


Fix it, then Ship it!





src/tests/authorization_tests.cpp
Lines 5379 (patched)


Remove extra period.


- Greg Mann


On Dec. 15, 2017, 7:07 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64657/
> ---
> 
> (Updated Dec. 15, 2017, 7:07 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added tests for VIEW_STANDALONE_CONTAINER ACL.
> 
> 
> Diffs
> -
> 
>   src/tests/authorization_tests.cpp 35eecc8970fae09ca38b322f0f2575a086b6ecd7 
> 
> 
> Diff: https://reviews.apache.org/r/64657/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64656: Added validation for VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64656/#review193986
---


Fix it, then Ship it!





src/authorizer/local/authorizer.cpp
Lines 1674 (patched)


s/container/containers/

It's too bad the other standalone container ACL field names are not 
plural...


- Greg Mann


On Dec. 15, 2017, 7:07 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64656/
> ---
> 
> (Updated Dec. 15, 2017, 7:07 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added validation for VIEW_STANDALONE_CONTAINER ACL.
> 
> 
> Diffs
> -
> 
>   src/authorizer/local/authorizer.cpp 
> 09f26185a14e563b79dedd66caaf323b6c3f9abc 
> 
> 
> Diff: https://reviews.apache.org/r/64656/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64655: Updated upgrades.md about the GET_CONTAINERS API change.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64655/#review193985
---


Fix it, then Ship it!





docs/upgrades.md
Lines 392 (patched)


Looks like the '#' is unnecessary?



docs/upgrades.md
Lines 394 (patched)


s/API/API call/


- Greg Mann


On Dec. 15, 2017, 6:26 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64655/
> ---
> 
> (Updated Dec. 15, 2017, 6:26 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated upgrades.md about the GET_CONTAINERS API change.
> 
> 
> Diffs
> -
> 
>   docs/upgrades.md 0c0b7d5623074733e185612e6f11779a5b17fb0e 
> 
> 
> Diff: https://reviews.apache.org/r/64655/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64639: Supported getting all containers in the agent API.

2017-12-15 Thread Jie Yu


> On Dec. 15, 2017, 9:46 p.m., Greg Mann wrote:
> > src/slave/http.cpp
> > Lines 2275-2276 (original), 2380-2384 (patched)
> > 
> >
> > Nit: these fit on one line.

The second one does not :( thus the change.


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64639/#review193965
---


On Dec. 15, 2017, 4:56 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64639/
> ---
> 
> (Updated Dec. 15, 2017, 4:56 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Supported getting all containers in the agent API.
> 
> 
> Diffs
> -
> 
>   src/internal/evolve.cpp 6ce6150ee60e8db6f6a0c8febefb781d5c1daf1f 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
>   src/slave/http.hpp 9ca061758da90f799fd012d3a9b717b8e3ad70f0 
>   src/slave/http.cpp ed22b9f6bfa1c480a0672ce25d364bba6e33a200 
>   src/tests/agent_container_api_tests.cpp 
> c2c15eb4812d8f45d54f96cf43bbf0c77a75578a 
> 
> 
> Diff: https://reviews.apache.org/r/64639/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64654: Updated CHANGELOG about the change to GET_CONTAINERS API.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64654/#review193983
---


Fix it, then Ship it!





CHANGELOG
Lines 20 (patched)


s/nested/nested & standalone/


- Greg Mann


On Dec. 15, 2017, 6:26 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64654/
> ---
> 
> (Updated Dec. 15, 2017, 6:26 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated CHANGELOG about the change to GET_CONTAINERS API.
> 
> 
> Diffs
> -
> 
>   CHANGELOG 8b3d3642c50a60b6ce0f9d27031c2077254473de 
> 
> 
> Diff: https://reviews.apache.org/r/64654/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64653: Added doc for new fields in GET_CONTAINERS operator API.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64653/#review193982
---


Ship it!




Ship It!

- Greg Mann


On Dec. 15, 2017, 6:25 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64653/
> ---
> 
> (Updated Dec. 15, 2017, 6:25 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added doc for new fields in GET_CONTAINERS operator API.
> 
> 
> Diffs
> -
> 
>   docs/operator-http-api.md fbe29143f50df5719662168cd6c655500988e6d0 
> 
> 
> Diff: https://reviews.apache.org/r/64653/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64628: Updated GET_CONTAINERS API allowing nested/standalone containers.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64628/#review193981
---


Ship it!




Ship It!

- Greg Mann


On Dec. 15, 2017, 4:54 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64628/
> ---
> 
> (Updated Dec. 15, 2017, 4:54 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Bugs: MESOS-8040
> https://issues.apache.org/jira/browse/MESOS-8040
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated GET_CONTAINERS API allowing nested/standalone containers.
> 
> 
> Diffs
> -
> 
>   include/mesos/agent/agent.proto c3b1f22b7e414d83d405d7d1a3b38b6fbfbfab79 
>   include/mesos/v1/agent/agent.proto 280a3b0975ec20a6a6fc2407e47a40ce4574fc9c 
> 
> 
> Diff: https://reviews.apache.org/r/64628/diff/3/
> 
> 
> Testing
> ---
> 
> n/a
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64639: Supported getting all containers in the agent API.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64639/#review193965
---


Fix it, then Ship it!





src/slave/http.cpp
Lines 2327 (patched)


s/bool/const bool/



src/slave/http.cpp
Lines 2331 (patched)


Nit:
"MesosContainerizer-specific methods"



src/slave/http.cpp
Lines 2341-2342 (patched)


Nit: fits on one line.



src/slave/http.cpp
Lines 2275-2276 (original), 2380-2384 (patched)


Nit: these fit on one line.



src/tests/agent_container_api_tests.cpp
Lines 426 (patched)


Looks like this param is only set to `true` currently, should we just set 
it in the function body like `show_standalone`?


- Greg Mann


On Dec. 15, 2017, 4:56 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64639/
> ---
> 
> (Updated Dec. 15, 2017, 4:56 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Supported getting all containers in the agent API.
> 
> 
> Diffs
> -
> 
>   src/internal/evolve.cpp 6ce6150ee60e8db6f6a0c8febefb781d5c1daf1f 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
>   src/slave/http.hpp 9ca061758da90f799fd012d3a9b717b8e3ad70f0 
>   src/slave/http.cpp ed22b9f6bfa1c480a0672ce25d364bba6e33a200 
>   src/tests/agent_container_api_tests.cpp 
> c2c15eb4812d8f45d54f96cf43bbf0c77a75578a 
> 
> 
> Diff: https://reviews.apache.org/r/64639/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64638: Added VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64638/#review193980
---


Ship it!




Ship It!

- Greg Mann


On Dec. 15, 2017, 4:55 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64638/
> ---
> 
> (Updated Dec. 15, 2017, 4:55 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added VIEW_STANDALONE_CONTAINER ACL.
> 
> 
> Diffs
> -
> 
>   include/mesos/authorizer/acls.proto 
> 40a1425ca51c5bb70f7af2e17d605f2125dcb4cb 
>   include/mesos/authorizer/authorizer.proto 
> 7db5fb32442f05db44f5e5dbaef9671be0288827 
>   src/authorizer/local/authorizer.cpp 
> 09f26185a14e563b79dedd66caaf323b6c3f9abc 
> 
> 
> Diff: https://reviews.apache.org/r/64638/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64658: Modified SLRP to use the VolumeProfileAdaptor module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64658/
---

(Updated Dec. 15, 2017, 9:46 p.m.)


Review request for mesos, Jie Yu and Joseph Wu.


Changes
---

Rebased and minor name changes.


Repository: mesos


Description
---

This changes the Storage Local Resource Provider's source of profile
information from only using the default to using a VolumeProfileAdaptor
module.

This patch is based on https://reviews.apache.org/r/64616.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
158b6b408002209aa9a79a6772da30c984aad61a 


Diff: https://reviews.apache.org/r/64658/diff/2/

Changes: https://reviews.apache.org/r/64658/diff/1-2/


Testing
---

See later in chain. This patch will fail SLRP tests. They are fixed in a 
followup patch.


Thanks,

Chun-Hung Hsiao



Re: Review Request 64591: Printed out the source profile and ID for a disk resource.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64591/
---

(Updated Dec. 15, 2017, 9:44 p.m.)


Review request for mesos and Jie Yu.


Changes
---

Addressed comments.


Repository: mesos


Description
---

`Resource::DiskInfo::Source` is printed out in the following formats:

  TYPE(profile,id):root

If the source has no profile and ID, then "(profile,id)" will be
skipped. If either the profile or the ID is missing, the missing part
will be replaced with an empty string. If the source has no root path,
":root" will be skipped.


Diffs (updated)
-

  src/common/resources.cpp 2774372bb52d9d60847a8230282068432b28b054 
  src/v1/resources.cpp 43d9b0f48c345b4574171cffbd1dc07bda1d1546 


Diff: https://reviews.apache.org/r/64591/diff/2/

Changes: https://reviews.apache.org/r/64591/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Re: Review Request 64621: Fixed a corner case for pre-existing volumes created by old RPs.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64621/
---

(Updated Dec. 15, 2017, 9:43 p.m.)


Review request for mesos, Jie Yu and Joseph Wu.


Changes
---

Addressed comments.


Repository: mesos


Description
---

When an agent failed over and registered as a new one, the RP will be
registered as a new one as well, but it could pick up the checkpointed
states for volumes created by an old RP as pre-existing volumes. We
should use the checkpointed capabilities for these volumes instead of
using the default mount or block capability.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
158b6b408002209aa9a79a6772da30c984aad61a 


Diff: https://reviews.apache.org/r/64621/diff/2/

Changes: https://reviews.apache.org/r/64621/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Re: Review Request 64522: Made OfferOperationStatusUpdateManager tests pass on Windows.

2017-12-15 Thread Gaston Kleiman

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/
---

(Updated Dec. 15, 2017, 1:25 p.m.)


Review request for mesos, Andrew Schwartzmeyer, Greg Mann, and Jie Yu.


Summary (updated)
-

Made OfferOperationStatusUpdateManager tests pass on Windows.


Repository: mesos


Description (updated)
---

See the summary.


Diffs
-

  src/status_update_manager/status_update_manager_process.hpp 
e575d8c31bb6bd63fc26f01e2deec5f44821e01c 
  src/tests/offer_operation_status_update_manager_tests.cpp 
cb71bc10f9a687f12ee4cca4f8fa8614f5ea2d58 


Diff: https://reviews.apache.org/r/64522/diff/10/


Testing (updated)
---

It took a while, but the Windows reviewbot is now happy =).


Thanks,

Gaston Kleiman



Re: Review Request 64522: WIP: try to make tests pass on Windows.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/#review193975
---



PASS: Mesos patch 64522 was successfully built and tested.

Reviews applied: `['64522']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522

- Mesos Reviewbot Windows


On Dec. 14, 2017, 10:31 p.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64522/
> ---
> 
> (Updated Dec. 14, 2017, 10:31 p.m.)
> 
> 
> Review request for mesos and Gaston Kleiman.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> WIP: try to make tests pass on Windows.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> e575d8c31bb6bd63fc26f01e2deec5f44821e01c 
>   src/tests/offer_operation_status_update_manager_tests.cpp 
> cb71bc10f9a687f12ee4cca4f8fa8614f5ea2d58 
> 
> 
> Diff: https://reviews.apache.org/r/64522/diff/10/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>



Re: Review Request 64659: Fixed unit tests for volume profile module integration.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64659/#review193970
---



FAIL: Failed to apply the dependent review: 64658.

Failed command: `python.exe .\support\apply-reviews.py -n -r 64658`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64659

Relevant logs:

- 
[apply-review-64658-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64659/logs/apply-review-64658-stdout.log):

```
error: patch failed: src/resource_provider/storage/provider.cpp:337
error: src/resource_provider/storage/provider.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Dec. 15, 2017, 11:45 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64659/
> ---
> 
> (Updated Dec. 15, 2017, 11:45 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Fixed unit tests for volume profile module integration.
> 
> 
> Diffs
> -
> 
>   src/module/manager.cpp d93648ab3f95e17d28056e9f5eb75ebff30775e1 
>   src/tests/agent_resource_provider_config_api_tests.cpp 
> 3c68b732a1d41561f98206a55ac09d91229f330a 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 1f11825001768864b61fdebdb24168790d7ec4fe 
> 
> 
> Diff: https://reviews.apache.org/r/64659/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 64644: Refactored and fixed bugs for SLRP resource reconciliation.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64644/
---

(Updated Dec. 15, 2017, 8:21 p.m.)


Review request for mesos, Jie Yu and Joseph Wu.


Changes
---

Addressed comments.


Repository: mesos


Description
---

Refactored and fixed bugs for SLRP resource reconciliation.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
158b6b408002209aa9a79a6772da30c984aad61a 


Diff: https://reviews.apache.org/r/64644/diff/2/

Changes: https://reviews.apache.org/r/64644/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Re: Review Request 64657: Added tests for VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64657/#review193969
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64628', '64638', '64639', '64653', '64654', '64655', 
'64656', '64657']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64657

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64657/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2376 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2400 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2372 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2396 ms total)

[--] Global test environment tear-down
[==] 836 tests from 85 test cases ran. (310974 ms total)
[  PASSED  ] 826 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 213 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64657/logs/mesos-tests-stderr.log):

```
I1215 20:20:35.306864  9580 master.cpp:10158] Updating the state of task 
3d5da677-c5e0-41d9-94ac-af1fe267cdee of framework 
7d86efd4-8ed7-449e-874f-ab52cd471258- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1215 20:20:35.307835  8928 slave.cpp:3401] Shutting down framework 
7d86efd4-8ed7-449e-874f-ab52cd471258-
I1215 20:20:35.307835  8928 slave.cpp:6109] Shutting down executor 
'3d5da677-c5e0-41d9-94ac-af1fI1215 20:20:34.624876  8448 exec.cpp:162] Version: 
1.5.0
I1215 20:20:34.649875   100 exec.cpp:237] Executor registered on agent 
7d86efd4-8ed7-449e-874f-ab52cd471258-S0
I1215 20:20:34.652901  8688 executor.cpp:171] Received SUBSCRIBED event
I1215 20:20:34.657873  8688 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 20:20:34.657873  8688 executor.cpp:171] Received LAUNCH event
I1215 20:20:34.661872  8688 executor.cpp:638] Starting task 
3d5da677-c5e0-41d9-94ac-af1fe267cdee
I1215 20:20:34.738867  8688 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 20:20:35.280838  8688 executor.cpp:651] Forked command at 2616
I1215 20:20:35.309836  7212 exec.cpp:435] Executor asked to shutdown
I1215 20:20:35.309836  8688 executor.cpp:171] Received SHUTDOWN event
I1215 20:20:35.309836  8688 executor.cpp:748] Shutting down
I1215 20:20:35.309836  8688 executor.cpp:855] Sending SIGTERM to process tree 
at pid 2e267cdee' of framework 7d86efd4-8ed7-449e-874f-ab52cd471258- at 
executor(1)@10.3.1.11:56430
I1215 20:20:35.308862  8928 slave.cpp:909] Agent terminating
W1215 20:20:35.308862  8928 slave.cpp:3397] Ignoring shutdown framework 
7d86efd4-8ed7-449e-874f-ab52cd471258- because it is terminating
I1215 20:20:35.309836  9580 master.cpp:10264] Removing task 
3d5da677-c5e0-41d9-94ac-af1fe267cdee with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 7d86efd4-8ed7-449e-874f-ab52cd471258- on 
agent 7d86efd4-8ed7-449e-874f-ab52cd471258-S0 at slave(327)@10.3.1.11:56409 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 20:20:35.310837  3132 containerizer.cpp:2352] Destroying container 
07b80261-62e9-4229-b099-c1cceaacf817 in RUNNING state
I1215 20:20:35.310837  3132 containerizer.cpp:2955] Transitioning the state of 
container 07b80261-62e9-4229-b099-c1cceaacf817 from RUNNING to DESTROYING
I1215 20:20:35.310837  9580 master.cpp:1305] Agent 
7d86efd4-8ed7-449e-874f-ab52cd471258-S0 at slave(327)@10.3.1.11:56409 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 20:20:35

Re: Review Request 64644: Refactored and fixed bugs for SLRP resource reconciliation.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64644/#review193968
---


Fix it, then Ship it!




Please also add some more comments about the recocile function


src/resource_provider/storage/provider.cpp
Lines 1048 (patched)


NOTE?


- Jie Yu


On Dec. 15, 2017, 10:06 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64644/
> ---
> 
> (Updated Dec. 15, 2017, 10:06 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Refactored and fixed bugs for SLRP resource reconciliation.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> 17acf1b430b81e31bd97b117a44477eebab7e6f4 
> 
> 
> Diff: https://reviews.apache.org/r/64644/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 64644: Refactored and fixed bugs for SLRP resource reconciliation.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64644/#review193967
---




src/resource_provider/storage/provider.cpp
Lines 1072 (patched)


hum, should be `!checkpointed.contains(uncoverted)`


- Jie Yu


On Dec. 15, 2017, 10:06 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64644/
> ---
> 
> (Updated Dec. 15, 2017, 10:06 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Refactored and fixed bugs for SLRP resource reconciliation.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> 17acf1b430b81e31bd97b117a44477eebab7e6f4 
> 
> 
> Diff: https://reviews.apache.org/r/64644/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 64591: Printed out the source profile and ID for a disk resource.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64591/#review193964
---


Fix it, then Ship it!





src/v1/resources.cpp
Lines 2124 (patched)


I'd prefer `id` to be at the front


- Jie Yu


On Dec. 14, 2017, 1:55 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64591/
> ---
> 
> (Updated Dec. 14, 2017, 1:55 a.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> `Resource::DiskInfo::Source` is printed out in the following formats:
> 
>   TYPE(profile,id):root
> 
> If the source has no profile and ID, then "(profile,id)" will be
> skipped. If either the profile or the ID is missing, the missing part
> will be replaced with an empty string. If the source has no root path,
> ":root" will be skipped.
> 
> 
> Diffs
> -
> 
>   src/common/resources.cpp 2774372bb52d9d60847a8230282068432b28b054 
>   src/v1/resources.cpp 43d9b0f48c345b4574171cffbd1dc07bda1d1546 
> 
> 
> Diff: https://reviews.apache.org/r/64591/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Review Request 64659: Fixed unit tests for volume profile module integration.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64659/
---

Review request for mesos, Jie Yu and Joseph Wu.


Repository: mesos


Description
---

Fixed unit tests for volume profile module integration.


Diffs
-

  src/module/manager.cpp d93648ab3f95e17d28056e9f5eb75ebff30775e1 
  src/tests/agent_resource_provider_config_api_tests.cpp 
3c68b732a1d41561f98206a55ac09d91229f330a 
  src/tests/storage_local_resource_provider_tests.cpp 
1f11825001768864b61fdebdb24168790d7ec4fe 


Diff: https://reviews.apache.org/r/64659/diff/1/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Review Request 64658: Modified SLRP to use the VolumeProfileAdaptor module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64658/
---

Review request for mesos, Jie Yu and Joseph Wu.


Repository: mesos


Description
---

This changes the Storage Local Resource Provider's source of profile
information from only using the default to using a VolumeProfileAdaptor
module.

This patch is based on https://reviews.apache.org/r/64616.


Diffs
-

  src/resource_provider/storage/provider.cpp 
17acf1b430b81e31bd97b117a44477eebab7e6f4 


Diff: https://reviews.apache.org/r/64658/diff/1/


Testing
---

See later in chain. This patch will fail SLRP tests. They are fixed in a 
followup patch.


Thanks,

Chun-Hung Hsiao



Re: Review Request 64621: Fixed a corner case for pre-existing volumes created by old RPs.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64621/#review193963
---


Fix it, then Ship it!





src/resource_provider/storage/provider.cpp
Lines 2448-2449 (original), 2445-2446 (patched)


When you do `profiles.at`, you assumes that the profile already exist? Can 
you add some CHECK if that's your assumption? Ditto other places.


- Jie Yu


On Dec. 14, 2017, 7:34 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64621/
> ---
> 
> (Updated Dec. 14, 2017, 7:34 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> When an agent failed over and registered as a new one, the RP will be
> registered as a new one as well, but it could pick up the checkpointed
> states for volumes created by an old RP as pre-existing volumes. We
> should use the checkpointed capabilities for these volumes instead of
> using the default mount or block capability.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> 17acf1b430b81e31bd97b117a44477eebab7e6f4 
> 
> 
> Diff: https://reviews.apache.org/r/64621/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 64354: Added tests for UriVolumeProfile module.

2017-12-15 Thread Joseph Wu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64354/
---

(Updated Dec. 15, 2017, 11:29 a.m.)


Review request for mesos, Chun-Hung Hsiao and Jie Yu.


Changes
---

Update test usage of `--uri` module flag.


Bugs: MESOS-8251
https://issues.apache.org/jira/browse/MESOS-8251


Repository: mesos


Description
---

These tests mostly exercise the parsing logic used by the module
and some of the interaction expected from callers of the module
(i.e. the Storage Local Resource Provider).


Diffs (updated)
-

  src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
  src/tests/volume_profile_tests.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/64354/diff/3/

Changes: https://reviews.apache.org/r/64354/diff/2-3/


Testing
---

make check


Thanks,

Joseph Wu



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Joseph Wu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/
---

(Updated Dec. 15, 2017, 11:29 a.m.)


Review request for mesos, Chun-Hung Hsiao and Jie Yu.


Changes
---

Fixed ability to load `file://` for the `--uri` flag correctly.  This also 
relaxed the flag a bit, so it is possible to specify a path like 
`/path/to/thing` rather than `file:///path/to/thing`.


Bugs: MESOS-8251
https://issues.apache.org/jira/browse/MESOS-8251


Repository: mesos


Description
---

This example module shows how a VolumeProfile module might be
implemented (and is a viable module in its own right).  The module
can be configured to fetch a map of profiles from a URI (`file://` or
`http(s)://`) and possibly cache this item for some time.


Diffs (updated)
-

  src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
  src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
  src/resource_provider/uri_volume_profile.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/64353/diff/5/

Changes: https://reviews.apache.org/r/64353/diff/4-5/


Testing
---

See end of chain.


Thanks,

Joseph Wu



Re: Review Request 64352: Added default VolumeProfile module implementation.

2017-12-15 Thread Joseph Wu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64352/
---

(Updated Dec. 15, 2017, 11:28 a.m.)


Review request for mesos, Chun-Hung Hsiao and Jie Yu.


Changes
---

Added some missing module manager init.


Bugs: MESOS-8251
https://issues.apache.org/jira/browse/MESOS-8251


Repository: mesos


Description
---

By default, Mesos and the Storage Local Resource Provider (SLRP) will
not expect profiles to be used. This is partly due to the experimental
nature of the "profile" field and partly because some explicit operator
intervention is required to get profiles defined. There is no default
profile that is accepted by CSI plugins.


Diffs (updated)
-

  include/mesos/module/volume_profile.hpp PRE-CREATION 
  src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
  src/module/manager.cpp d93648ab3f95e17d28056e9f5eb75ebff30775e1 
  src/resource_provider/volume_profile.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/64352/diff/3/

Changes: https://reviews.apache.org/r/64352/diff/2-3/


Testing
---

See end of chain.


Thanks,

Joseph Wu



Re: Review Request 64655: Updated upgrades.md about the GET_CONTAINERS API change.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64655/#review193955
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64628', '64638', '64639', '64653', '64654', '64655']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64655

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64655/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2497 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2521 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2376 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2401 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (312307 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 213 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64655/logs/mesos-tests-stderr.log):

```
I1215 19:20:23.621690   612 master.cpp:10158] Updating the state of task 
a492ff27-c270-43f4-ab5d-5adf3402d380 of framework 
f4a2004c-f72e-4a93-a106-4f81701e5b54- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1215 19:20:23.621690  8796 slave.cpp:3401] Shutting down framework 
f4a2004c-f72e-4a93I1215 19:20:22.937747  7948 exec.cpp:162] Version: 1.5.0
I1215 19:20:22.960743  1912 exec.cpp:237] Executor registered on agent 
f4a2004c-f72e-4a93-a106-4f81701e5b54-S0
I1215 19:20:22.963749  5460 executor.cpp:171] Received SUBSCRIBED event
I1215 19:20:22.967746  5460 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 19:20:22.967746  5460 executor.cpp:171] Received LAUNCH event
I1215 19:20:22.972748  5460 executor.cpp:638] Starting task 
a492ff27-c270-43f4-ab5d-5adf3402d380
I1215 19:20:23.054740  5460 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 19:20:23.594683  5460 executor.cpp:651] Forked command at 6056
I1215 19:20:23.623682  9892 exec.cpp:435] Executor asked to shutdown
I1215 19:20:23.624682  5460 executor.cpp:171] Received SHUTDOWN event
I1215 19:20:23.624682  5460 executor.cpp:748] Shutting down
I1215 19:20:23.625682  5460 executor.cpp:855] Sending SIGTERM to process tree 
at pid 6-a106-4f81701e5b54-
I1215 19:20:23.621690  8796 slave.cpp:6109] Shutting down executor 
'a492ff27-c270-43f4-ab5d-5adf3402d380' of framework 
f4a2004c-f72e-4a93-a106-4f81701e5b54- at executor(1)@10.3.1.11:54496
I1215 19:20:23.622683  8796 slave.cpp:909] Agent terminating
W1215 19:20:23.622683  8796 slave.cpp:3397] Ignoring shutdown framework 
f4a2004c-f72e-4a93-a106-4f81701e5b54- because it is terminating
I1215 19:20:23.623682   612 master.cpp:10264] Removing task 
a492ff27-c270-43f4-ab5d-5adf3402d380 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework f4a2004c-f72e-4a93-a106-4f81701e5b54- on 
agent f4a2004c-f72e-4a93-a106-4f81701e5b54-S0 at slave(327)@10.3.1.11:54475 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 19:20:23.625682  3084 containerizer.cpp:2352] Destroying container 
e9fd6e8d-91e8-49cf-b50e-1264acd00be1 in RUNNING state
I1215 19:20:23.626682  3084 containerizer.cpp:2955] Transitioning the state of 
container e9fd6e8d-91e8-49cf-b50e-1264acd00be1 from RUNNING to DESTROYING
I1215 19:20:23.626682   612 master.cpp:1305] Agent 
f4a2004c-f72e-4a93-a106-4f81701e5b54-S0 at slave(327)@10.3.1.11:54475 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 19:20:23.626682   612 master

Review Request 64656: Added validation for VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64656/
---

Review request for mesos, Gilbert Song and Greg Mann.


Repository: mesos


Description
---

Added validation for VIEW_STANDALONE_CONTAINER ACL.


Diffs
-

  src/authorizer/local/authorizer.cpp 09f26185a14e563b79dedd66caaf323b6c3f9abc 


Diff: https://reviews.apache.org/r/64656/diff/1/


Testing
---

make check


Thanks,

Jie Yu



Review Request 64657: Added tests for VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64657/
---

Review request for mesos, Gilbert Song and Greg Mann.


Repository: mesos


Description
---

Added tests for VIEW_STANDALONE_CONTAINER ACL.


Diffs
-

  src/tests/authorization_tests.cpp 35eecc8970fae09ca38b322f0f2575a086b6ecd7 


Diff: https://reviews.apache.org/r/64657/diff/1/


Testing
---

make check


Thanks,

Jie Yu



Review Request 64655: Updated upgrades.md about the GET_CONTAINERS API change.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64655/
---

Review request for mesos, Gilbert Song and Greg Mann.


Repository: mesos


Description
---

Updated upgrades.md about the GET_CONTAINERS API change.


Diffs
-

  docs/upgrades.md 0c0b7d5623074733e185612e6f11779a5b17fb0e 


Diff: https://reviews.apache.org/r/64655/diff/1/


Testing
---

make check


Thanks,

Jie Yu



Review Request 64654: Updated CHANGELOG about the change to GET_CONTAINERS API.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64654/
---

Review request for mesos, Gilbert Song and Greg Mann.


Repository: mesos


Description
---

Updated CHANGELOG about the change to GET_CONTAINERS API.


Diffs
-

  CHANGELOG 8b3d3642c50a60b6ce0f9d27031c2077254473de 


Diff: https://reviews.apache.org/r/64654/diff/1/


Testing
---

make check


Thanks,

Jie Yu



Review Request 64653: Added doc for new fields in GET_CONTAINERS operator API.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64653/
---

Review request for mesos, Gilbert Song and Greg Mann.


Repository: mesos


Description
---

Added doc for new fields in GET_CONTAINERS operator API.


Diffs
-

  docs/operator-http-api.md fbe29143f50df5719662168cd6c655500988e6d0 


Diff: https://reviews.apache.org/r/64653/diff/1/


Testing
---

make check


Thanks,

Jie Yu



Re: Review Request 64639: Supported getting all containers in the agent API.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64639/#review193942
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64623', '64624', '64628', '64638', '64639']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64639

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64639/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2443 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2469 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2356 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2381 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (308956 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 213 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64639/logs/mesos-tests-stderr.log):

```
I1215 17:50:17.010546  7136 master.cpp:10160] Updating the state of task 
06ff5e85-43d4-4327-aecf-3053f7d0122b of framework 
ae54db67-236e-48e6-92d7-dde1dd725ef8- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1215 17:50:17.010546  1328 slave.cpp:3401] Shutting down framework 
ae54db67-236e-48e6-92d7-dde1dd725ef8-
I1215 17:50:17.010546  1328 slave.cpp:6109] Shutting down executor 
'06ff5e85-43d4-4327-aecf-3053f7d0122b' of framework 
ae54db67-236e-48e6-92d7-dde1dd725ef8- at executor(1)@10.3.1.11:52537
I1215 17:50:17.011545  1328 I1215 17:50:16.314610  3916 exec.cpp:162] Version: 
1.5.0
I1215 17:50:16.340603  1684 exec.cpp:237] Executor registered on agent 
ae54db67-236e-48e6-92d7-dde1dd725ef8-S0
I1215 17:50:16.343626  3760 executor.cpp:171] Received SUBSCRIBED event
I1215 17:50:16.348611  3760 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 17:50:16.348611  3760 executor.cpp:171] Received LAUNCH event
I1215 17:50:16.352600  3760 executor.cpp:638] Starting task 
06ff5e85-43d4-4327-aecf-3053f7d0122b
I1215 17:50:16.438592  3760 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 17:50:16.983548  3760 executor.cpp:651] Forked command at 4148
I1215 17:50:17.011545  5924 exec.cpp:435] Executor asked to shutdown
I1215 17:50:17.012545  3760 executor.cpp:171] Received SHUTDOWN event
I1215 17:50:17.012545  3760 executor.cpp:748] Shutting down
I1215 17:50:17.012545  3760 executor.cpp:855] Sending SIGTERM to process tree 
at pid 4slave.cpp:909] Agent terminating
W1215 17:50:17.012545  1328 slave.cpp:3397] Ignoring shutdown framework 
ae54db67-236e-48e6-92d7-dde1dd725ef8- because it is terminating
I1215 17:50:17.012545  7136 master.cpp:10266] Removing task 
06ff5e85-43d4-4327-aecf-3053f7d0122b with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework ae54db67-236e-48e6-92d7-dde1dd725ef8- on 
agent ae54db67-236e-48e6-92d7-dde1dd725ef8-S0 at slave(327)@10.3.1.11:52514 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 17:50:17.015548  4304 containerizer.cpp:2352] Destroying container 
413416d6-3694-4042-84f9-46668e4e9aaa in RUNNING state
I1215 17:50:17.015548  7136 master.cpp:1305] Agent 
ae54db67-236e-48e6-92d7-dde1dd725ef8-S0 at slave(327)@10.3.1.11:52514 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 17:50:17.015548  4304 containerizer.cpp:2955] Transitioning the state of 
container 413416d6-3694-4042-84f9-46668e4e9aaa from RUNNING to DESTROYING
I1215 17:50:17.015548  7136 master.cpp:336

Re: Review Request 64600: Fixed handling of resource provider updates without resources.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64600/#review193935
---


Ship it!




Ship It!

- Jie Yu


On Dec. 14, 2017, 10:01 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64600/
> ---
> 
> (Updated Dec. 14, 2017, 10:01 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> On master failover the master might learn about a resource provider
> without resources. This patch relaxes an assertion in the master
> permitting this scenarios.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp 2f2608f5d1369971cabd0d69cb4f1a2aaea8c1dd 
> 
> 
> Diff: https://reviews.apache.org/r/64600/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>



Re: Review Request 64639: Supported getting all containers in the agent API.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64639/
---

(Updated Dec. 15, 2017, 4:56 p.m.)


Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.


Changes
---

Rebased.


Repository: mesos


Description
---

Supported getting all containers in the agent API.


Diffs (updated)
-

  src/internal/evolve.cpp 6ce6150ee60e8db6f6a0c8febefb781d5c1daf1f 
  src/slave/containerizer/mesos/containerizer.cpp 
7ab0b07f689f872573ca458ae47cd6426ebc0365 
  src/slave/http.hpp 9ca061758da90f799fd012d3a9b717b8e3ad70f0 
  src/slave/http.cpp ed22b9f6bfa1c480a0672ce25d364bba6e33a200 
  src/tests/agent_container_api_tests.cpp 
c2c15eb4812d8f45d54f96cf43bbf0c77a75578a 


Diff: https://reviews.apache.org/r/64639/diff/2/

Changes: https://reviews.apache.org/r/64639/diff/1-2/


Testing
---

make check


Thanks,

Jie Yu



Re: Review Request 64638: Added VIEW_STANDALONE_CONTAINER ACL.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64638/
---

(Updated Dec. 15, 2017, 4:55 p.m.)


Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.


Repository: mesos


Description
---

Added VIEW_STANDALONE_CONTAINER ACL.


Diffs (updated)
-

  include/mesos/authorizer/acls.proto 40a1425ca51c5bb70f7af2e17d605f2125dcb4cb 
  include/mesos/authorizer/authorizer.proto 
7db5fb32442f05db44f5e5dbaef9671be0288827 
  src/authorizer/local/authorizer.cpp 09f26185a14e563b79dedd66caaf323b6c3f9abc 


Diff: https://reviews.apache.org/r/64638/diff/2/

Changes: https://reviews.apache.org/r/64638/diff/1-2/


Testing
---

make check


Thanks,

Jie Yu



Re: Review Request 64628: Updated GET_CONTAINERS API allowing nested/standalone containers.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64628/
---

(Updated Dec. 15, 2017, 4:54 p.m.)


Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.


Changes
---

Fix a merging error.


Bugs: MESOS-8040
https://issues.apache.org/jira/browse/MESOS-8040


Repository: mesos


Description
---

Updated GET_CONTAINERS API allowing nested/standalone containers.


Diffs (updated)
-

  include/mesos/agent/agent.proto c3b1f22b7e414d83d405d7d1a3b38b6fbfbfab79 
  include/mesos/v1/agent/agent.proto 280a3b0975ec20a6a6fc2407e47a40ce4574fc9c 


Diff: https://reviews.apache.org/r/64628/diff/3/

Changes: https://reviews.apache.org/r/64628/diff/2-3/


Testing
---

n/a


Thanks,

Jie Yu



Re: Review Request 64650: Made resource provider driver start explicit.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64650/#review193933
---


Ship it!




Ship It!

- Jie Yu


On Dec. 15, 2017, 1:29 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64650/
> ---
> 
> (Updated Dec. 15, 2017, 1:29 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao, Jie Yu, and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The driver for HTTP connections of resource providers is implemented
> as an actor which takes callbacks. It previously started listening for
> and handling of events on creation. We typically store drivers as
> member of resource providers.
> 
> This setup is in general problematic since it e.g., becomes impossible
> to write safe callbacks using internal resource provider state; it was
> e.g., impossible to write a safe callback using the member driver to
> send calls. This is due to a race between initialization of the member
> driver and a newly created driver starting to handle events and
> triggering callbacks making use of a not yet initialized member driver
> variable. As a concrete example, with a 'MockResourceProvider' holding
> a 'std::unique_ptr' member tests would regularly hit this
> race, even though the time window between creating a temporary to
> assign to the member and the assignment appeared to be small.
> 
> This patch introduces an explicit 'start' method to the driver which
> is to be used to explicitly start processing of events by the driver.
> 
> 
> Diffs
> -
> 
>   include/mesos/v1/resource_provider.hpp 
> 006889a090d5d211cf8e03d67c849753639207f3 
>   src/resource_provider/driver.cpp 62c4ca11e40fae7c2da929511ce304908e4e9256 
>   src/resource_provider/http_connection.hpp 
> 207120a6d28d231276b34ce30e6ae2f640884e19 
>   src/resource_provider/storage/provider.cpp 
> 17acf1b430b81e31bd97b117a44477eebab7e6f4 
>   src/tests/mesos.hpp 6207d621f0496c69f34280da2246d7d9967ba55b 
> 
> 
> Diff: https://reviews.apache.org/r/64650/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> Without this patch 
> `ContentType/ResourceProviderManagerHttpApiTest.ConvertResources/0` fails 
> after < 30 iterations; with this patch it passes > 2000 iterations.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>



Re: Review Request 64648: Fixed handling of resource versions in agent oversubscribed updates.

2017-12-15 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64648/#review193932
---


Ship it!




Ship It!

- Jie Yu


On Dec. 15, 2017, 1:15 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64648/
> ---
> 
> (Updated Dec. 15, 2017, 1:15 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Agents can use 'UpdateSlaveMessage' to send updates on their
> oversubscribed resources, their resource provider state, or both. We
> previously assumed that 'UpdateSlaveMessage' from a resource
> provider-capable agent would always contain the most recent resource
> version of the agent even thought the field is marked 'optional'.
> 
> This patch simplifies the handling in the master to not assert a set
> resource version for 'UpdateSlaveMessage' for resource
> provider-capable agents. Instead we explicitly and unconditionally
> check whether the field is set and handle only set values.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp e082da8267fa22c26818c67bd6da573fe1808696 
> 
> 
> Diff: https://reviews.apache.org/r/64648/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on a number of platforms and setups in internal CI.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>



Re: Review Request 64069: Ensured command executor always honors shutdown request.

2017-12-15 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64069/#review193931
---




src/launcher/executor.cpp
Lines 768 (patched)


Can you add a test for this?


- Vinod Kone


On Dec. 15, 2017, 10:43 a.m., Alexander Rukletsov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64069/
> ---
> 
> (Updated Dec. 15, 2017, 10:43 a.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Anand Mazumdar, Armand Grillet, and 
> Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> See summary.
> 
> 
> Diffs
> -
> 
>   src/launcher/executor.cpp c688c04e598ac140421fd5e47359b0e48d30bcc5 
> 
> 
> Diff: https://reviews.apache.org/r/64069/diff/1/
> 
> 
> Testing
> ---
> 
> See https://reviews.apache.org/r/64033/
> 
> 
> Thanks,
> 
> Alexander Rukletsov
> 
>



Re: Review Request 64628: Updated GET_CONTAINERS API allowing nested/standalone containers.

2017-12-15 Thread Jie Yu


> On Dec. 15, 2017, 1:01 p.m., Gilbert Song wrote:
> > include/mesos/agent/agent.proto
> > Line 349 (original), 355 (patched)
> > 
> >
> > Missed `optional GetContainers get_containers = 20;` here?

err... my mistake. It's in https://reviews.apache.org/r/64639. Let me fix it


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64628/#review193909
---


On Dec. 15, 2017, 4:05 a.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64628/
> ---
> 
> (Updated Dec. 15, 2017, 4:05 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Bugs: MESOS-8040
> https://issues.apache.org/jira/browse/MESOS-8040
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated GET_CONTAINERS API allowing nested/standalone containers.
> 
> 
> Diffs
> -
> 
>   include/mesos/agent/agent.proto c3b1f22b7e414d83d405d7d1a3b38b6fbfbfab79 
>   include/mesos/v1/agent/agent.proto 280a3b0975ec20a6a6fc2407e47a40ce4574fc9c 
> 
> 
> Diff: https://reviews.apache.org/r/64628/diff/2/
> 
> 
> Testing
> ---
> 
> n/a
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64650: Made resource provider driver start explicit.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64650/#review193914
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64650']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64650

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64650/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2280 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2303 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2373 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2401 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (311940 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 205 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64650/logs/mesos-tests-stderr.log):

```
I1215 15:10:34.312325  3196 master.cpp:10160] Updating the state of task 
4132dc43-9dd7-4636-97fc-767bcd315ef0 of framework 
06edae8c-bb37-40cc-ad12-026263f09cd3- (latest state: TASK_KILLED, status 
update I1215 15:10:33.663337  9916 exec.cpp:162] Version: 1.5.0
I1215 15:10:33.685343  7424 exec.cpp:237] Executor registered on agent 
06edae8c-bb37-40cc-ad12-026263f09cd3-S0
I1215 15:10:33.688338  8708 executor.cpp:171] Received SUBSCRIBED event
I1215 15:10:33.692342  8708 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 15:10:33.693341  8708 executor.cpp:171] Received LAUNCH event
I1215 15:10:33.697350  8708 executor.cpp:638] Starting task 
4132dc43-9dd7-4636-97fc-767bcd315ef0
I1215 15:10:33.774338  8708 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 15:10:34.285323  8708 executor.cpp:651] Forked command at 8940
I1215 15:10:34.314326  3796 exec.cpp:435] Executor asked to shutdown
I1215 15:10:34.315325  9968 executor.cpp:171] Received SHUTDOWN event
I1215 15:10:34.315325  9968 executor.cpp:748] Shutting down
I1215 15:10:34.315325  9968 executor.cpp:855] Sending SIGTERM to process tree 
at pid 8state: TASK_KILLED)
I1215 15:10:34.312325  3944 slave.cpp:3401] Shutting down framework 
06edae8c-bb37-40cc-ad12-026263f09cd3-
I1215 15:10:34.312325  3944 slave.cpp:6109] Shutting down executor 
'4132dc43-9dd7-4636-97fc-767bcd315ef0' of framework 
06edae8c-bb37-40cc-ad12-026263f09cd3- at executor(1)@10.3.1.11:50472
I1215 15:10:34.314326  3944 slave.cpp:909] Agent terminating
W1215 15:10:34.314326  3944 slave.cpp:3397] Ignoring shutdown framework 
06edae8c-bb37-40cc-ad12-026263f09cd3- because it is terminating
I1215 15:10:34.314326  3196 master.cpp:10266] Removing task 
4132dc43-9dd7-4636-97fc-767bcd315ef0 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 06edae8c-bb37-40cc-ad12-026263f09cd3- on 
agent 06edae8c-bb37-40cc-ad12-026263f09cd3-S0 at slave(327)@10.3.1.11:50451 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 15:10:34.317353  3196 master.cpp:1305] Agent 
06edae8c-bb37-40cc-ad12-026263f09cd3-S0 at slave(327)@10.3.1.11:50451 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 15:10:34.317353  3196 master.cpp:3364] Disconnecting agent 
06edae8c-bb37-40cc-ad12-026263f09cd3-S0 at slave(327)@10.3.1.11:50451 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 15:10:34.317353  8756 containerizer.cpp:2337] Destroying container 
7b1b72fd-ac34-43a0-a576-156fee96aafd in RUNNING state
I1215 15:10:34.317353  3288 h

Re: Review Request 64648: Fixed handling of resource versions in agent oversubscribed updates.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64648/#review193912
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64648']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64648

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64648/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2425 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2450 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2374 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2400 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (319626 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 205 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64648/logs/mesos-tests-stderr.log):

```
I1215 14:11:51.022387  6896 slave.cpp:3401] Shutting down framework 
1f3c1bf2-6079-4fd6-937f-984bfd86e2ce-
I1215 14:11:51.022387  6896 slave.cpp:6109] Shutting down executor 
'891703b9-fd6f-4f81-b804-5d47fc675f0c' of framI1215 14:11:50.339382  7592 
exec.cpp:162] Version: 1.5.0
I1215 14:11:50.363382  3216 exec.cpp:237] Executor registered on agent 
1f3c1bf2-6079-4fd6-937f-984bfd86e2ce-S0
I1215 14:11:50.366389  1476 executor.cpp:171] Received SUBSCRIBED event
I1215 14:11:50.371387  1476 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 14:11:50.371387  1476 executor.cpp:171] Received LAUNCH event
I1215 14:11:50.375386  1476 executor.cpp:638] Starting task 
891703b9-fd6f-4f81-b804-5d47fc675f0c
I1215 14:11:50.454388  1476 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 14:11:50.997388  1476 executor.cpp:651] Forked command at 7272
I1215 14:11:51.025388  9772 exec.cpp:435] Executor asked to shutdown
I1215 14:11:51.025388  1476 executor.cpp:171] Received SHUTDOWN event
I1215 14:11:51.025388  1476 executor.cpp:748] Shutting down
I1215 14:11:51.025388  1476 executor.cpp:855] Sending SIGTERM to process tree 
at pid 7ework 1f3c1bf2-6079-4fd6-937f-984bfd86e2ce- at 
executor(1)@10.3.1.11:64964
I1215 14:11:51.023658  4796 master.cpp:10156] Updating the state of task 
891703b9-fd6f-4f81-b804-5d47fc675f0c of framework 
1f3c1bf2-6079-4fd6-937f-984bfd86e2ce- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1215 14:11:51.024386  6896 slave.cpp:909] Agent terminating
W1215 14:11:51.024386  6896 slave.cpp:3397] Ignoring shutdown framework 
1f3c1bf2-6079-4fd6-937f-984bfd86e2ce- because it is terminating
I1215 14:11:51.025388  4796 master.cpp:10262] Removing task 
891703b9-fd6f-4f81-b804-5d47fc675f0c with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 1f3c1bf2-6079-4fd6-937f-984bfd86e2ce- on 
agent 1f3c1bf2-6079-4fd6-937f-984bfd86e2ce-S0 at slave(327)@10.3.1.11:64942 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 14:11:51.026387  8100 containerizer.cpp:2337] Destroying container 
99967f28-5cd5-483e-b317-47b4dfbf6fb7 in RUNNING state
I1215 14:11:51.027398  8100 containerizer.cpp:2939] Transitioning the state of 
container 99967f28-5cd5-483e-b317-47b4dfbf6fb7 from RUNNING to DESTROYING
I1215 14:11:51.028406  4796 master.cpp:1305] Agent 
1f3c1bf2-6079-4fd6-937f-984bfd86e2ce-S0 at slave(327)@10.3.1.11:64942 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 14:11:51.028406  4796 master.cpp:3364] Disconnecting agent 
1f3c1bf2-607

Re: Review Request 64650: Made resource provider driver start explicit.

2017-12-15 Thread Jan Schlicht

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64650/#review193911
---


Ship it!




Ship It!

- Jan Schlicht


On Dec. 15, 2017, 2:29 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64650/
> ---
> 
> (Updated Dec. 15, 2017, 2:29 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao, Jie Yu, and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The driver for HTTP connections of resource providers is implemented
> as an actor which takes callbacks. It previously started listening for
> and handling of events on creation. We typically store drivers as
> member of resource providers.
> 
> This setup is in general problematic since it e.g., becomes impossible
> to write safe callbacks using internal resource provider state; it was
> e.g., impossible to write a safe callback using the member driver to
> send calls. This is due to a race between initialization of the member
> driver and a newly created driver starting to handle events and
> triggering callbacks making use of a not yet initialized member driver
> variable. As a concrete example, with a 'MockResourceProvider' holding
> a 'std::unique_ptr' member tests would regularly hit this
> race, even though the time window between creating a temporary to
> assign to the member and the assignment appeared to be small.
> 
> This patch introduces an explicit 'start' method to the driver which
> is to be used to explicitly start processing of events by the driver.
> 
> 
> Diffs
> -
> 
>   include/mesos/v1/resource_provider.hpp 
> 006889a090d5d211cf8e03d67c849753639207f3 
>   src/resource_provider/driver.cpp 62c4ca11e40fae7c2da929511ce304908e4e9256 
>   src/resource_provider/http_connection.hpp 
> 207120a6d28d231276b34ce30e6ae2f640884e19 
>   src/resource_provider/storage/provider.cpp 
> 17acf1b430b81e31bd97b117a44477eebab7e6f4 
>   src/tests/mesos.hpp 6207d621f0496c69f34280da2246d7d9967ba55b 
> 
> 
> Diff: https://reviews.apache.org/r/64650/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> Without this patch 
> `ContentType/ResourceProviderManagerHttpApiTest.ConvertResources/0` fails 
> after < 30 iterations; with this patch it passes > 2000 iterations.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>



Review Request 64650: Made resource provider driver start explicit.

2017-12-15 Thread Benjamin Bannier

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64650/
---

Review request for mesos, Chun-Hung Hsiao, Jie Yu, and Jan Schlicht.


Repository: mesos


Description
---

The driver for HTTP connections of resource providers is implemented
as an actor which takes callbacks. It previously started listening for
and handling of events on creation. We typically store drivers as
member of resource providers.

This setup is in general problematic since it e.g., becomes impossible
to write safe callbacks using internal resource provider state; it was
e.g., impossible to write a safe callback using the member driver to
send calls. This is due to a race between initialization of the member
driver and a newly created driver starting to handle events and
triggering callbacks making use of a not yet initialized member driver
variable. As a concrete example, with a 'MockResourceProvider' holding
a 'std::unique_ptr' member tests would regularly hit this
race, even though the time window between creating a temporary to
assign to the member and the assignment appeared to be small.

This patch introduces an explicit 'start' method to the driver which
is to be used to explicitly start processing of events by the driver.


Diffs
-

  include/mesos/v1/resource_provider.hpp 
006889a090d5d211cf8e03d67c849753639207f3 
  src/resource_provider/driver.cpp 62c4ca11e40fae7c2da929511ce304908e4e9256 
  src/resource_provider/http_connection.hpp 
207120a6d28d231276b34ce30e6ae2f640884e19 
  src/resource_provider/storage/provider.cpp 
17acf1b430b81e31bd97b117a44477eebab7e6f4 
  src/tests/mesos.hpp 6207d621f0496c69f34280da2246d7d9967ba55b 


Diff: https://reviews.apache.org/r/64650/diff/1/


Testing
---

`make check`

Without this patch 
`ContentType/ResourceProviderManagerHttpApiTest.ConvertResources/0` fails after 
< 30 iterations; with this patch it passes > 2000 iterations.


Thanks,

Benjamin Bannier



Review Request 64648: Fixed handling of resource versions in agent oversubscribed updates.

2017-12-15 Thread Benjamin Bannier

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64648/
---

Review request for mesos, Jie Yu and Jan Schlicht.


Repository: mesos


Description
---

Agents can use 'UpdateSlaveMessage' to send updates on their
oversubscribed resources, their resource provider state, or both. We
previously assumed that 'UpdateSlaveMessage' from a resource
provider-capable agent would always contain the most recent resource
version of the agent even thought the field is marked 'optional'.

This patch simplifies the handling in the master to not assert a set
resource version for 'UpdateSlaveMessage' for resource
provider-capable agents. Instead we explicitly and unconditionally
check whether the field is set and handle only set values.


Diffs
-

  src/master/master.cpp e082da8267fa22c26818c67bd6da573fe1808696 


Diff: https://reviews.apache.org/r/64648/diff/1/


Testing
---

Tested on a number of platforms and setups in internal CI.


Thanks,

Benjamin Bannier



Re: Review Request 64628: Updated GET_CONTAINERS API allowing nested/standalone containers.

2017-12-15 Thread Gilbert Song

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64628/#review193909
---




include/mesos/agent/agent.proto
Line 349 (original), 355 (patched)


Missed `optional GetContainers get_containers = 20;` here?


- Gilbert Song


On Dec. 14, 2017, 8:05 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64628/
> ---
> 
> (Updated Dec. 14, 2017, 8:05 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, Joseph Wu, and Vinod Kone.
> 
> 
> Bugs: MESOS-8040
> https://issues.apache.org/jira/browse/MESOS-8040
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Updated GET_CONTAINERS API allowing nested/standalone containers.
> 
> 
> Diffs
> -
> 
>   include/mesos/agent/agent.proto c3b1f22b7e414d83d405d7d1a3b38b6fbfbfab79 
>   include/mesos/v1/agent/agent.proto 280a3b0975ec20a6a6fc2407e47a40ce4574fc9c 
> 
> 
> Diff: https://reviews.apache.org/r/64628/diff/2/
> 
> 
> Testing
> ---
> 
> n/a
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64647: Updated tests related to containerizer refactoring.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64647/#review193908
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64646', '64647']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64647

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64647/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2347 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2371 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2387 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2412 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (310277 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 205 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64647/logs/mesos-tests-stderr.log):

```
I1215 12:39:59.617643   932 master.cpp:10160] Updating the state of task 
53c795f0-ad32-4dd4-909f-455fe21f5a56 of framework 
e01de3d2-6354-424f-8488-41321d48a48b- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1215 12:39:59.617643  2500 slave.cpp:3401] Shutting down framework 
e01de3d2-6354-424f-8488-41321d48a48b-
I1215 12:39:59.617643  2500 slave.cpp:6109] Shutting down executor 
'53c795f0-ad32-4dd4-909f-455fe21f5a56' of framework 
e01de3d2-6354-424f-8488-41321d48a48b- at executor(1)@10.3.1.11:63008
I1215 12:39I1215 12:39:58.958652  3876 exec.cpp:162] Version: 1.5.0
I1215 12:39:58.982682  6908 exec.cpp:237] Executor registered on agent 
e01de3d2-6354-424f-8488-41321d48a48b-S0
I1215 12:39:58.985682  4776 executor.cpp:171] Received SUBSCRIBED event
I1215 12:39:58.990675  4776 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1215 12:39:58.990675  4776 executor.cpp:171] Received LAUNCH event
I1215 12:39:58.994653  4776 executor.cpp:638] Starting task 
53c795f0-ad32-4dd4-909f-455fe21f5a56
I1215 12:39:59.069672  4776 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1215 12:39:59.592644  4776 executor.cpp:651] Forked command at 5760
I1215 12:39:59.619763  3808 exec.cpp:435] Executor asked to shutdown
I1215 12:39:59.620645  2768 executor.cpp:171] Received SHUTDOWN event
I1215 12:39:59.620645  2768 executor.cpp:748] Shutting down
I1215 12:39:59.620645  2768 executor.cpp:855] Sending SIGTERM to process tree 
at pid 5:59.618643  2500 slave.cpp:909] Agent terminating
W1215 12:39:59.618643  2500 slave.cpp:3397] Ignoring shutdown framework 
e01de3d2-6354-424f-8488-41321d48a48b- because it is terminating
I1215 12:39:59.619763   932 master.cpp:10266] Removing task 
53c795f0-ad32-4dd4-909f-455fe21f5a56 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework e01de3d2-6354-424f-8488-41321d48a48b- on 
agent e01de3d2-6354-424f-8488-41321d48a48b-S0 at slave(327)@10.3.1.11:62987 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1215 12:39:59.621644  2980 containerizer.cpp:2338] Destroying container 
0dcb73de-50b5-487e-9549-f61ee34010f8 in RUNNING state
I1215 12:39:59.622643  2980 containerizer.cpp:2945] Transitioning the state of 
container 0dcb73de-50b5-487e-9549-f61ee34010f8 from RUNNING to DESTROYING
I1215 12:39:59.622643   932 master.cpp:1305] Agent 
e01de3d2-6354-424f-8488-41321d48a48b-S0 at slave(327)@10.3.1.11:62987 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1215 12:39:59.622643   932 master.cpp:3364] Disconnecting agent 
e01d

Re: Review Request 64624: Added standalone isolator capability.

2017-12-15 Thread Gilbert Song

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64624/#review193905
---


Ship it!




LGTM!

- Gilbert Song


On Dec. 14, 2017, 12:19 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64624/
> ---
> 
> (Updated Dec. 14, 2017, 12:19 p.m.)
> 
> 
> Review request for mesos and Joseph Wu.
> 
> 
> Bugs: MESOS-7302
> https://issues.apache.org/jira/browse/MESOS-7302
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Recently, we introduced the standalone container suport (MESOS-7302).
> Some isolators might not be able to handle standalone containers.
> Therefore, we introduce an isolator capability (similar to nesting
> capability) so that we won't try to invoke those isolators that do not
> support standalone containers when dealing with a standalone container.
> 
> 
> Diffs
> -
> 
>   include/mesos/slave/isolator.hpp c52563fa055ae4d949f40db53c0f2eaec1cf9036 
>   src/slave/containerizer/mesos/containerizer.hpp 
> 965e183bb5c54f31d90e910edd35313ab380cea9 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
>   src/slave/containerizer/mesos/io/switchboard.hpp 
> 520a6ef7a888ad350909ca60adbb493970e971b7 
>   src/slave/containerizer/mesos/io/switchboard.cpp 
> af8fbe8ed6e90bfaa650fa5c0ac5fa655e6e3afd 
>   src/slave/containerizer/mesos/isolator.hpp 
> aaad346fce68b1fae1d84331bade358ebfe3db87 
>   src/slave/containerizer/mesos/isolator.cpp 
> ccdcb9a3710f26dbc23042d70cae3fcf4cbd527f 
>   src/slave/containerizer/mesos/isolators/appc/runtime.hpp 
> 8d225878ddbd04e4297b84c9ff782a994986aa4e 
>   src/slave/containerizer/mesos/isolators/appc/runtime.cpp 
> 535ea1adb6dc8f9444aeb760a7c69cacc9e4bc38 
>   src/slave/containerizer/mesos/isolators/cgroups/cgroups.hpp 
> 229bafc088c1c8f46384b8853516c0ebfce8593e 
>   src/slave/containerizer/mesos/isolators/cgroups/cgroups.cpp 
> c0ebc4e988a56d954e6c5831d931967331251f32 
>   src/slave/containerizer/mesos/isolators/docker/runtime.hpp 
> 642dd53a8028d26b0a8c7429a502d33db26838b7 
>   src/slave/containerizer/mesos/isolators/docker/runtime.cpp 
> 93394f899b2a53c7dd4f129e205351e7194a65f2 
>   src/slave/containerizer/mesos/isolators/docker/volume/isolator.hpp 
> 2e64d055e146719363a4118c0a63779524f7d9e5 
>   src/slave/containerizer/mesos/isolators/docker/volume/isolator.cpp 
> 6efdc75315dc68a73687c9b22ffe2c38136ca51f 
>   src/slave/containerizer/mesos/isolators/environment_secret.hpp 
> b98e8fe0d297d6c813a68e1d85681957d6577eb4 
>   src/slave/containerizer/mesos/isolators/environment_secret.cpp 
> 5b0b2fc7e0667c5ba4fabcd45330d9e55a35fb6a 
>   src/slave/containerizer/mesos/isolators/filesystem/linux.hpp 
> 6584d1e1cef410861df7b2dfd4006f1bb8a71698 
>   src/slave/containerizer/mesos/isolators/filesystem/linux.cpp 
> aa939e34abdb4b1dd98a0c32ce1e39ae76e062b8 
>   src/slave/containerizer/mesos/isolators/gpu/isolator.hpp 
> f3103a7ea2ad937ca1968faa2b8c6aa390ab6237 
>   src/slave/containerizer/mesos/isolators/gpu/isolator.cpp 
> 7b664267bf7f39552c8b7aea95a966845de966f1 
>   src/slave/containerizer/mesos/isolators/linux/capabilities.hpp 
> b9862a23490f1383742e6539c158efc2962497de 
>   src/slave/containerizer/mesos/isolators/linux/capabilities.cpp 
> 21d851e33e69b35a46d262276c2a87a256dd0fd9 
>   src/slave/containerizer/mesos/isolators/namespaces/ipc.hpp 
> 98504079e336aa4f9c6dd8cf8bd0690e79e19f29 
>   src/slave/containerizer/mesos/isolators/namespaces/ipc.cpp 
> 2d89d598d24e3bcf01d652ce3f586c9e3ccfc20b 
>   src/slave/containerizer/mesos/isolators/namespaces/pid.hpp 
> 6b4ba06c1067910a48fad4309e5fce156f6cd808 
>   src/slave/containerizer/mesos/isolators/namespaces/pid.cpp 
> 4f8253b58018581e022eb1832b9b07703cbd318d 
>   src/slave/containerizer/mesos/isolators/posix/disk.hpp 
> 68d9744de8fbff60f23dbd59bc242ad5f95256f6 
>   src/slave/containerizer/mesos/isolators/posix/disk.cpp 
> eb2302516d6e77873c22f279375e7659126b289c 
>   src/slave/containerizer/mesos/isolators/posix/rlimits.hpp 
> 0bce083a17e221400f1426ee64c87c3b48dac10a 
>   src/slave/containerizer/mesos/isolators/posix/rlimits.cpp 
> 0136652d03c987416b725e2a596358c714561110 
>   src/slave/containerizer/mesos/isolators/volume/host_path.hpp 
> 08c3d5f6a8643172d5cb3a629161e8aea9f0e4b5 
>   src/slave/containerizer/mesos/isolators/volume/host_path.cpp 
> 3f3f28068a4d1e68f5bfd4ba97ef9193845581a4 
>   src/slave/containerizer/mesos/isolators/volume/image.hpp 
> 8465bb5e14c6450e7dd6c82411fa9fdeeb517aa8 
>   src/slave/containerizer/mesos/isolators/volume/image.cpp 
> 15e242d5d6550928fae9d583a26fd02f9df3973f 
>   src/slave/containerizer/mesos/isolators/volume/sandbox_path.hpp 
> 12fab61855cdbdbd4bbbdad2335722448a5b7b07 
>   src/slave/containerizer/mesos/isolat

Review Request 64647: Updated tests related to containerizer refactoring.

2017-12-15 Thread Andrei Budnik

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64647/
---

Review request for mesos, Alexander Rukletsov, Gilbert Song, and Jie Yu.


Repository: mesos


Description
---

This patch is needed due to changes in Containerizer interface,
where `wait()`, `destroy()` and `kill()` methods return the same
ContainerTermination type.


Diffs
-

  src/tests/containerizer.hpp d6a69d6327018065453f8a6cb275d921d10c9198 
  src/tests/containerizer.cpp 13a290ff60e359ceced95b20cb9db8c25cd59dbc 
  src/tests/containerizer/composing_containerizer_tests.cpp 
7c22f162b128c3fdf8d4b20cac73fdf442449d79 
  src/tests/containerizer/docker_containerizer_tests.cpp 
630bb2e352dd55ecb401730c84f823e0a1f2d310 
  src/tests/containerizer/io_switchboard_tests.cpp 
f217d02bba2d1372392ad4da521940d27e4f69d4 
  src/tests/containerizer/mesos_containerizer_tests.cpp 
0729464a7a9b290c8634536e58909f9b38b58a9c 
  src/tests/containerizer/mock_containerizer.hpp 
01c617a55899a7125ca1dc80091aef5603419ee8 


Diff: https://reviews.apache.org/r/64647/diff/1/


Testing
---

make check (fedora 25)
internal CI


Thanks,

Andrei Budnik



Review Request 64646: Unified return type of `wait`, `destroy`, `kill` containerizer methods.

2017-12-15 Thread Andrei Budnik

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64646/
---

Review request for mesos, Alexander Rukletsov, Gilbert Song, and Jie Yu.


Repository: mesos


Description
---

This change refactors `wait()`, `destroy()` and `kill()` methods of
Containerizer base class as well as MesosContainerizer,
DockerContainerizer and ComposingContainerizer in order to return the
same ContainerTermination type. In fact, all these methods depend on
the same container termination promise, so this unification makes both
containerizer interface and internal logic more consistent.


Diffs
-

  src/slave/containerizer/composing.hpp 
1221d9b61b7cfbdbaa560dbafb0158225331ed6d 
  src/slave/containerizer/composing.cpp 
9ace70d9fbd78182715c5ef13fcaf7ad45f76f97 
  src/slave/containerizer/containerizer.hpp 
836283aa8aa827459558d567e090a230c32351ed 
  src/slave/containerizer/docker.hpp f1b56c8edced17e9786c86adb9b92c829ed81635 
  src/slave/containerizer/docker.cpp 890cb2aba44fe76e891117833eac8ccca00b759b 
  src/slave/containerizer/mesos/containerizer.hpp 
965e183bb5c54f31d90e910edd35313ab380cea9 
  src/slave/containerizer/mesos/containerizer.cpp 
7ab0b07f689f872573ca458ae47cd6426ebc0365 
  src/slave/http.cpp ed22b9f6bfa1c480a0672ce25d364bba6e33a200 


Diff: https://reviews.apache.org/r/64646/diff/1/


Testing
---

make check (fedora 25)
internal CI


Thanks,

Andrei Budnik



Re: Review Request 64379: Improved logs displayed after a slave failed recovery.

2017-12-15 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64379/#review193903
---


Ship it!




Ship It!

- Benno Evers


On Dec. 13, 2017, 7:26 p.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64379/
> ---
> 
> (Updated Dec. 13, 2017, 7:26 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Benno Evers.
> 
> 
> Bugs: MESOS-8328
> https://issues.apache.org/jira/browse/MESOS-8328
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Add some steps to clean the Docker daemon
> state used by the Docker containerizer.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp d997b4272578efffed05d38771f17df387ccac48 
> 
> 
> Diff: https://reviews.apache.org/r/64379/diff/4/
> 
> 
> Testing
> ---
> 
> New logs:
> ```
> E1213 10:58:10.826020 10057 slave.cpp:6738] EXIT with status 1: Failed to 
> perform recovery: 
> If recovery failed due to a change in configuration and you want to
> keep the current agent id, you might want to change the
> `--reconfiguration_policy` flag to a more permissive value.
> 
> To restart this agent with a new agent id instead, do as follows:
> rm -f /tmp/agent/meta/slaves/latest
> This ensures that the agent does not recover old live executors.
> 
> If you use the Docker containerizer and think that the Docker
> daemon state is broken, you can try to clear it. But be careful:
> these commands will erase all containers and images from this host,
> not just those started by Mesos!
> docker kill $(docker ps -q)
> docker rm $(docker ps -a -q)
> docker rmi $(docker images -q)
> 
> Finally, restart the agent.
> ```
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 64069: Ensured command executor always honors shutdown request.

2017-12-15 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64069/
---

(Updated Dec. 15, 2017, 10:43 a.m.)


Review request for mesos, Andrei Budnik, Anand Mazumdar, Armand Grillet, and 
Vinod Kone.


Repository: mesos


Description
---

See summary.


Diffs
-

  src/launcher/executor.cpp c688c04e598ac140421fd5e47359b0e48d30bcc5 


Diff: https://reviews.apache.org/r/64069/diff/1/


Testing
---

See https://reviews.apache.org/r/64033/


Thanks,

Alexander Rukletsov



Re: Review Request 64574: Ensured trailing '/' in URL is insignificant.

2017-12-15 Thread Alexander Rukletsov


> On Dec. 14, 2017, 9:40 p.m., Benjamin Mahler wrote:
> > Can you link to some prior art? What do the RFCs say recommend, if 
> > anything? What do other http frameworks do?

According to [RFC3986](https://tools.ietf.org/html/rfc3986#section-3.3), every 
'/' is meaningful, i.e., `/path` and `/path/` have different number of 
segments: ["path"] vs. ["path", ""]. However in practice it is recommended to 
treat paths with leading and trailing slashes in the same way to avoid user 
confusion, see [google's 
recommendation](https://webmasters.googleblog.com/2010/04/to-slash-or-not-to-slash.html).
 It is also recommended to redirect one request to another to avoid duplicate 
content, but I don't think it is relevant for us.


- Alexander


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64574/#review193848
---


On Dec. 13, 2017, 1:39 p.m., Alexander Rukletsov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64574/
> ---
> 
> (Updated Dec. 13, 2017, 1:39 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Benjamin Mahler.
> 
> 
> Bugs: MESOS-5333
> https://issues.apache.org/jira/browse/MESOS-5333
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Prior to this patch, adding a trailing '/' to a valid URL path, e.g.,
> "/state/", yielded a 404 response. This patch ensures that two URLs
> which differ only in trailing '/' produce the same result.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/src/process.cpp 
> 75cf1d3b6d3d257ba9bc81c68017a74a6511cebf 
>   3rdparty/libprocess/src/tests/http_tests.cpp 
> 9daac715f0242921b7f9f5c20b3eb27f1be802d4 
> 
> 
> Diff: https://reviews.apache.org/r/64574/diff/1/
> 
> 
> Testing
> ---
> 
> Ensured the modified test fails without the fix.
> `make check` on Mac OS 10.11.6 and several Linux distributions.
> 
> 
> Thanks,
> 
> Alexander Rukletsov
> 
>



Re: Review Request 64575: Added containerClass() to MesosContainerizerProcess::Container.

2017-12-15 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64575/#review193896
---




src/slave/containerizer/mesos/containerizer.hpp
Lines 358-359 (patched)


Please backtick class values.


- Alexander Rukletsov


On Dec. 13, 2017, 7:47 p.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64575/
> ---
> 
> (Updated Dec. 13, 2017, 7:47 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Bugs: MESOS-7361
> https://issues.apache.org/jira/browse/MESOS-7361
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This function always returns a ContainerClass, `DEFAULT` being the
> default value returned. Also simplifies the conditional statements
> in mesos/containerizer.cpp to use the function added.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.hpp 
> 965e183bb5c54f31d90e910edd35313ab380cea9 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
> 
> 
> Diff: https://reviews.apache.org/r/64575/diff/2/
> 
> 
> Testing
> ---
> 
> ```
> GTEST_FILTER="" nice make check -j16 V=0
> ```
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 63953: Added logging based on container class.

2017-12-15 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63953/#review193897
---


Fix it, then Ship it!





src/slave/containerizer/mesos/containerizer.hpp
Lines 54-55 (patched)


Please backtick class values.

Also, let's explain why we do it (to avoid polluting the agent log with 
DEBUG container related information, because DEBUG containers can be started in 
a periodic manner).



src/slave/containerizer/mesos/containerizer.cpp
Line 2335 (original), 2338-2339 (patched)


Ups


- Alexander Rukletsov


On Dec. 13, 2017, 7:47 p.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63953/
> ---
> 
> (Updated Dec. 13, 2017, 7:47 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Bugs: MESOS-7361
> https://issues.apache.org/jira/browse/MESOS-7361
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This change adjusts log level based on the container class.
> If the class is `DEBUG` we print the log entry at a verbose
> level 1, otherwise we print it at the `INFO` level.
> 
> We use the added macro in mesos containerizer so that COMMAND
> checks cause less INFO logs (15 lines instead of 26 before).
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.hpp 
> 965e183bb5c54f31d90e910edd35313ab380cea9 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
> 
> 
> Diff: https://reviews.apache.org/r/63953/diff/8/
> 
> 
> Testing
> ---
> 
> Started a Mesos cluster and used `mesos-execute` with this task group to 
> check that the behaviour after this patch is the one expected:
> 
> ```
> {
>   "tasks": [
> {
>   "name": "Name of the task",
>   "task_id": {
> "value": "task-group"
>   },
>   "agent_id": {
> "value": ""
>   },
>   "resources": [
> {
>   "name": "cpus",
>   "type": "SCALAR",
>   "scalar": {
> "value": 0.01
>   }
> },
> {
>   "name": "mem",
>   "type": "SCALAR",
>   "scalar": {
> "value": 2
>   }
> }
>   ],
>   "command": {
> "value": "sleep 1000"
>   },
>   "check": {
> "type": "COMMAND",
> "command": {
>   "command": {
> "value": "echo \"Bonjour\""
>   },
>   "uris": []
> }
>   }
> }
>   ]
> }
> ```
> 
> And:
> ```
> $ nice make check
> ```
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 64596: WIP: Added unit tests for storage local resource provider recovery.

2017-12-15 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64596/#review193895
---



FAIL: Failed to apply the dependent review: 64644.

Failed command: `python.exe .\support\apply-reviews.py -n -r 64644`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64596

Relevant logs:

- 
[apply-review-64644-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64596/logs/apply-review-64644-stdout.log):

```
error: patch failed: src/resource_provider/storage/provider.cpp:1008
error: src/resource_provider/storage/provider.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Dec. 14, 2017, 3:44 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64596/
> ---
> 
> (Updated Dec. 14, 2017, 3:44 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Joseph Wu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> WIP: Added unit tests for storage local resource provider recovery.
> 
> 
> Diffs
> -
> 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 1f11825001768864b61fdebdb24168790d7ec4fe 
> 
> 
> Diff: https://reviews.apache.org/r/64596/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Review Request 64644: Refactored and fixed bugs for SLRP resource reconciliation.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64644/
---

Review request for mesos, Jie Yu and Joseph Wu.


Repository: mesos


Description
---

Refactored and fixed bugs for SLRP resource reconciliation.


Diffs
-

  src/resource_provider/storage/provider.cpp 
17acf1b430b81e31bd97b117a44477eebab7e6f4 


Diff: https://reviews.apache.org/r/64644/diff/1/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao



Re: Review Request 64623: Fixed a bug in handling standalone container recovery.

2017-12-15 Thread Gilbert Song

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64623/#review193894
---


Ship it!




Ship It!

- Gilbert Song


On Dec. 14, 2017, 12:14 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64623/
> ---
> 
> (Updated Dec. 14, 2017, 12:14 p.m.)
> 
> 
> Review request for mesos and Joseph Wu.
> 
> 
> Bugs: MESOS-7302
> https://issues.apache.org/jira/browse/MESOS-7302
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It's possible that the pid is not known for a standalone container. For
> instance, the agent crashes after the containerizer checkpoints the
> runtime directory, but before it is able to checkpoint the pid. In that
> case, we just assume that the child will exit due to the broken control
> pipe and we can move to destroy without recovering it.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
> 
> 
> Diff: https://reviews.apache.org/r/64623/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>



Re: Review Request 64575: Added containerClass() to MesosContainerizerProcess::Container.

2017-12-15 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64575/#review193893
---


Fix it, then Ship it!





src/slave/containerizer/mesos/containerizer.hpp
Lines 360 (patched)


This is not quite true. Even if `config` is not optional anymore, 
`ContainerConfig.container_class` might not be set and default to `0`. Please 
kill this line.



src/slave/containerizer/mesos/containerizer.cpp
Lines 2877-2884 (patched)


Since this function can be used not only in the launch path where we have 
the container, you should check `config` is not `None`. How about something 
like this then?

```
return (config.isSome() && config->has_container_class())
  ? config->container_class()
  : ContainerClass::DEFAULT;
``


- Alexander Rukletsov


On Dec. 13, 2017, 7:47 p.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64575/
> ---
> 
> (Updated Dec. 13, 2017, 7:47 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Bugs: MESOS-7361
> https://issues.apache.org/jira/browse/MESOS-7361
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This function always returns a ContainerClass, `DEFAULT` being the
> default value returned. Also simplifies the conditional statements
> in mesos/containerizer.cpp to use the function added.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.hpp 
> 965e183bb5c54f31d90e910edd35313ab380cea9 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 7ab0b07f689f872573ca458ae47cd6426ebc0365 
> 
> 
> Diff: https://reviews.apache.org/r/64575/diff/2/
> 
> 
> Testing
> ---
> 
> ```
> GTEST_FILTER="" nice make check -j16 V=0
> ```
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 64353: Added example VolumeProfile module.

2017-12-15 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64353/#review193892
---




src/resource_provider/uri_volume_profile.hpp
Lines 81 (patched)


We cannot do this through `Flags` because the content of the file referred 
to by the URI will be loaded: 
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/flags/fetch.hpp


- Chun-Hung Hsiao


On Dec. 9, 2017, 2:17 a.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64353/
> ---
> 
> (Updated Dec. 9, 2017, 2:17 a.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jie Yu.
> 
> 
> Bugs: MESOS-8251
> https://issues.apache.org/jira/browse/MESOS-8251
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This example module shows how a VolumeProfile module might be
> implemented (and is a viable module in its own right).  The module
> can be configured to fetch a map of profiles from a URI (`file://` or
> `http(s)://`) and possibly cache this item for some time.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am f5a4edd245e8d6535502e951ecaa526c2bae25f9 
>   src/resource_provider/uri_volume_profile.hpp PRE-CREATION 
>   src/resource_provider/uri_volume_profile.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64353/diff/4/
> 
> 
> Testing
> ---
> 
> See end of chain.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>