date:20180524


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67059/#review203732
---



Patch looks great!

Reviews applied: [67247, 67099, 67059]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 22, 2018, 3:55 p.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67059/
> ---
> 
> (Updated May 22, 2018, 3:55 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Eric Chung, 
> and Kevin Klues.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The scripts are in a temporary directory, support/python3.
> 
> The scripts have been ported using 2to3, the official tool to do so.
> Many of these scripts require testing from the community before being
> used by default.
> 
> The script building the virtual environment and the git hooks have
> been updated to use the new scripts if the environment variable
> `MESOSSUPPORTPYTHON` is set to `3` by the user.
> 
> 
> Diffs
> -
> 
>   support/README.md e812e9638ca555c56e4c52e52485240e75cb8229 
>   support/build-virtualenv 850af89326721f34de20eb45a7e78fa391d031be 
>   support/hooks/post-rewrite 1ab14abf711d1923a7ae69beb33581317009a94a 
>   support/hooks/pre-commit 6faba98ab6db68aef1a54091a08b8db1eaac8701 
>   support/python3/apply-reviews.py PRE-CREATION 
>   support/python3/generate-endpoint-help.py PRE-CREATION 
>   support/python3/jsonurl.py PRE-CREATION 
>   support/python3/mesos-gtest-runner.py PRE-CREATION 
>   support/python3/mesos-split.py PRE-CREATION 
>   support/python3/mesos-style.py PRE-CREATION 
>   support/python3/post-reviews.py PRE-CREATION 
>   support/python3/push-commits.py PRE-CREATION 
>   support/python3/test-upgrade.py PRE-CREATION 
>   support/python3/verify-reviews.py PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67059/diff/3/
> 
> 
> Testing
> ---
> 
> All the files are OK for our linter, I have tested `mesos-style.py` and 
> `post-reviews.py` (used for this review request).
> 
> We will likely see error messages `TypeError: cannot use a string pattern on 
> a bytes-like object`, they are very easy to fix (we just need to add 
> `.decode("utf-8")` but detecting all of them requires to use the scripts. 
> Please do so and create issues.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>

Re: Review Request 66668: Removed `destroyed` from `Container` struct in composing containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8/#review203734
---


Ship it!




Ship It!

- Qian Zhang


On April 17, 2018, 11:23 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8/
> ---
> 
> (Updated April 17, 2018, 11:23 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8712
> https://issues.apache.org/jira/browse/MESOS-8712
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously, we stored `destroyed` promise for each container and used
> it to guarantee that `destroy()` returns a non-empty value when the
> destroy-in-progress stops an launch-in-progress using the next
> containerizer. Since `wait()` and `destroy()` return the same
> `ContainerTermination` value when called with the same ContainerID
> argument, we can remove `destroyed` promise and add callbacks to
> clean up `containers_` map instead.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/composing.cpp 
> 1fb79f53b48154ecbd3b0165b6a8002c871e6dce 
> 
> 
> Diff: https://reviews.apache.org/r/8/diff/8/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 66668: Removed `destroyed` from `Container` struct in composing containerizer.



> On May 24, 2018, 4:24 p.m., Qian Zhang wrote:
> > Ship It!

A minor comment, since we fixed a tech debt (memory leak) in 
`ComposingContainerizerProcess::__recover()` in this patch, can you please 
mention it in the commit message?


- Qian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8/#review203734
---


On April 17, 2018, 11:23 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8/
> ---
> 
> (Updated April 17, 2018, 11:23 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8712
> https://issues.apache.org/jira/browse/MESOS-8712
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously, we stored `destroyed` promise for each container and used
> it to guarantee that `destroy()` returns a non-empty value when the
> destroy-in-progress stops an launch-in-progress using the next
> containerizer. Since `wait()` and `destroy()` return the same
> `ContainerTermination` value when called with the same ContainerID
> argument, we can remove `destroyed` promise and add callbacks to
> clean up `containers_` map instead.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/composing.cpp 
> 1fb79f53b48154ecbd3b0165b6a8002c871e6dce 
> 
> 
> Diff: https://reviews.apache.org/r/8/diff/8/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 66670: Ensured that `wait()` and `destroy()` return the same result.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66670/#review203736
---


Ship it!




Ship It!

- Qian Zhang


On April 17, 2018, 11:23 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66670/
> ---
> 
> (Updated April 17, 2018, 11:23 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8713
> https://issues.apache.org/jira/browse/MESOS-8713
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We need to return the same `ContainerTermination` result for both
> `wait()` and `destroy()` for a terminated container. This patch
> ensures that for a terminated nested container `destroy()` returns
> the same result as for `wait()`.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/composing.cpp 
> 1fb79f53b48154ecbd3b0165b6a8002c871e6dce 
>   src/slave/containerizer/mesos/containerizer.cpp 
> eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
> 
> 
> Diff: https://reviews.apache.org/r/66670/diff/5/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 66671: Updated composing containerizer tests.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66671/#review203738
---


Ship it!




Ship It!

- Qian Zhang


On April 17, 2018, 11:23 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66671/
> ---
> 
> (Updated April 17, 2018, 11:23 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8737
> https://issues.apache.org/jira/browse/MESOS-8737
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch updates composing containerizer tests in order to be
> consistent with the unification of `destroy()` and `wait()` return
> types.
> 
> 
> Diffs
> -
> 
>   src/tests/containerizer/composing_containerizer_tests.cpp 
> 6964ac2f050396a733b04e650e59d6c29c59665d 
> 
> 
> Diff: https://reviews.apache.org/r/66671/diff/5/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 65505: Restored `WaitAfterDestroy` test for a nested container.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65505/#review203739
---


Ship it!




Ship It!

- Qian Zhang


On April 27, 2018, 1 a.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65505/
> ---
> 
> (Updated April 27, 2018, 1 a.m.)
> 
> 
> Review request for mesos, Greg Mann, Jie Yu, Joseph Wu, Kevin Klues, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8734
> https://issues.apache.org/jira/browse/MESOS-8734
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test was removed in fd4b9af147, but it's important to check that
> after termination of a nested container, its termination status is
> available. This property is used in default executor.
> 
> 
> Diffs
> -
> 
>   src/tests/containerizer/nested_mesos_containerizer_tests.cpp 
> 661be81d6036113295876546f565264b843b704d 
> 
> 
> Diff: https://reviews.apache.org/r/65505/diff/4/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 67274: Rejected non-zero scalar resource values which be represented as zero.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67274/
---

(Updated May 24, 2018, 11:09 a.m.)


Review request for mesos, Jie Yu, Joris Van Remoortere, Michael Park, and Neil 
Conway.


Changes
---

More generally reject incorrectly rounded scalar resource values.


Summary (updated)
-

Rejected non-zero scalar resource values which be represented as zero.


Bugs: MESOS-8945
https://issues.apache.org/jira/browse/MESOS-8945


Repository: mesos


Description (updated)
---

Internally values of scalar resources are stored in a fixed point
representation. This can lead to (expected) precision loss.

This patch adds resource validation ensuring that values of scalar
resources are either zero or large enough so that they are still
represented as non-zero values after conversion to the internal fixed
point format.

While this patch should not change valid, intended use cases, it might
change the behavior for erroneous, already broken framework workflows.


Diffs (updated)
-

  src/common/resources.cpp 192d086e60b81c0ae8e26d4120beee2034f63d3e 
  src/tests/master_tests.cpp 6a0fa988aebaf1cf66abaaca0c628f981c6ccc31 
  src/tests/resources_tests.cpp 1201efec3ff0abc268f896c1fe2330571390b3fd 


Diff: https://reviews.apache.org/r/67274/diff/2/

Changes: https://reviews.apache.org/r/67274/diff/1-2/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Re: Review Request 67274: Rejected non-zero scalar resource values which be represented as zero.



> On May 24, 2018, 12:39 a.m., Jie Yu wrote:
> > src/common/validation.hpp
> > Lines 57 (patched)
> > 
> >
> > I wonder if we should just chagne `Resources::validate` given that we 
> > already check if resource.scalar is not negative in the method.

I initially put a check for non-zeroness into `Resources::validate`, but we do 
depend on being able to e.g., parse zero resource scalar amounts. As an 
example, agent default resources of the form `cpus:1;mem:32;disk:32;gpus:3` 
should remain allowed as they might be generated automatically; a similar 
argument could be made for resources explicitly specified as exactly zero in 
offer operations.

I relaxed the check to only reject non-zero resource values which would be 
rounded to zero and put it into `Resources::validate` instead.


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67274/#review203707
---


On May 24, 2018, 11:09 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67274/
> ---
> 
> (Updated May 24, 2018, 11:09 a.m.)
> 
> 
> Review request for mesos, Jie Yu, Joris Van Remoortere, Michael Park, and 
> Neil Conway.
> 
> 
> Bugs: MESOS-8945
> https://issues.apache.org/jira/browse/MESOS-8945
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Internally values of scalar resources are stored in a fixed point
> representation. This can lead to (expected) precision loss.
> 
> This patch adds resource validation ensuring that values of scalar
> resources are either zero or large enough so that they are still
> represented as non-zero values after conversion to the internal fixed
> point format.
> 
> While this patch should not change valid, intended use cases, it might
> change the behavior for erroneous, already broken framework workflows.
> 
> 
> Diffs
> -
> 
>   src/common/resources.cpp 192d086e60b81c0ae8e26d4120beee2034f63d3e 
>   src/tests/master_tests.cpp 6a0fa988aebaf1cf66abaaca0c628f981c6ccc31 
>   src/tests/resources_tests.cpp 1201efec3ff0abc268f896c1fe2330571390b3fd 
> 
> 
> Diff: https://reviews.apache.org/r/67274/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67137: Avoided leaking file descriptors in Mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67137/#review203744
---



Patch looks great!

Reviews applied: [67137]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 24, 2018, 6:25 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67137/
> ---
> 
> (Updated May 24, 2018, 6:25 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, Jie Yu, and James Peach.
> 
> 
> Bugs: MESOS-8917
> https://issues.apache.org/jira/browse/MESOS-8917
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Avoided leaking file descriptors in Mesos containerizer.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/launch.cpp 
> f25d90651ef32495c9161c3eaed8a327d1b2b926 
> 
> 
> Diff: https://reviews.apache.org/r/67137/diff/7/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> I still need to confirm this patch in CI on a wider range of scenarios.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67274: Rejected non-zero scalar resource values which be represented as zero.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67274/#review203745
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67274']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67274

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67274/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (107 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1045 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (38 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (72 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (735 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (760 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (712 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (735 ms total)

[--] Global test environment tear-down
[==] 982 tests from 95 test cases ran. (39 ms total)
[  PASSED  ] 981 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67274/logs/mesos-tests-stderr.log):

```
I0524 09:56:35.784303 18944 master.cpp:10843] Updating the state of task 
7b0031cc-2ed6-4a10-80ed-a0fd879a6b60 of framework 
d2fe544a-5061-4827-8faa-b24dbdf55d75- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 09:56:35.784303  9420 slave.cpp:3935] Shutting down framework 
d2fe544a-5061-4827-8faa-b24dbdf55d75-
I0524 09:56:35.784303  9420 slave.cpp:6656] Shutting down executor 
'7b0031cc-2ed6-4a10-80ed-a0fd879a6b60' of frI0524 09:56:35.623260  8052 
exec.cpp:162] Version: 1.7.0
I0524 09:56:35.647274 19548 exec.cpp:236] Executor registered on agent 
d2fe544a-5061-4827-8faa-b24dbdf55d75-S0
I0524 09:56:35.651271 15232 executor.cpp:178] Received SUBSCRIBED event
I0524 09:56:35.656299 15232 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 09:56:35.656299 15232 executor.cpp:178] Received LAUNCH event
I0524 09:56:35.661299 15232 executor.cpp:665] Starting task 
7b0031cc-2ed6-4a10-80ed-a0fd879a6b60
I0524 09:56:35.741295 15232 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 09:56:35.759310 15232 executor.cpp:678] Forked command at 12552
I0524 09:56:35.786283 17228 exec.cpp:445] Executor asked to shutdown
I0524 09:56:35.787279   584 executor.cpp:178] Received SHUTDOWN event
I0524 09:56:35.787279   584 executor.cpp:781] Shutting down
I0524 09:56:35.787279   584 executor.cpp:894] Sending SIGTERM to process tree 
at pid 12amework d2fe544a-5061-4827-8faa-b24dbdf55d75- at 
executor(1)@192.10.1.6:60198
I0524 09:56:35.786283 18944 master.cpp:10942] Removing task 
7b0031cc-2ed6-4a10-80ed-a0fd879a6b60 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework d2fe544a-5061-4827-8faa-b24dbdf55d75- on 
agent d2fe544a-5061-4827-8faa-b24dbdf55d75-S0 at slave(448)@192.10.1.6:60177 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 09:56:35.786283  9420 slave.cpp:929] Agent terminating
W0524 09:56:35.787279  9420 slave.cpp:3931] Ignoring shutdown framework 
d2fe544a-5061-4827-8faa-b24dbdf55d75- because it is terminating
I0524 09:56:35.789276 18476 master.cpp:1293] Agent 
d2fe544a-5061-4827-8faa-b24dbdf55d75-S0 at slave(448)@192.10.1.6:60177 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 09:56:35.789276 18476 master.cpp:3303] Disconnecting agent 
d2fe544a-5061-4827-8faa-b24dbdf55d75-S0 at slave(448)@192.10.1.6:60177 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 09:56:35.790284 16864 hierarchical.cpp:344] Removed framework 
d2fe544a-5061-4827-8faa-b24dbdf55d75-
I0524 09:56:35.790284 18476 master.cpp:3322] Deactivating agent 
d2fe544a-5061-4827-8faa-b24dbdf55d75-S0 at slave(448)@

Re: Review Request 67270: Fixed compilation error in libprocess example.

2018-05-24 Thread Benno Evers



> On May 23, 2018, 6:39 p.m., Benjamin Bannier wrote:
> > Could you follow this up with a patch which compiles the example at least 
> > as part of `check`?
> > 
> > For cmake we could e.g., add a `CMakeLists.txt` to the `example/` directory 
> > with just
> > 
> > add_executable(example EXCLUDE_FROM_ALL example.cpp)
> > target_link_libraries(example process)
> > add_dependencies(libprocess-tests example)
> > 
> > and add the subdirectory to `libprocess/CMakeLists.txt`.
> > 
> > Similar for autotools.
> > 
> > Maybe we also should format the file according to our style guide in 
> > another patch.

Sorry, I don't really have the time right now to work on that. But I'll 
volunteer to review the patch if you want to write it.


> On May 23, 2018, 6:39 p.m., Benjamin Bannier wrote:
> > 3rdparty/libprocess/examples/example.cpp
> > Line 68 (original), 68 (patched)
> > 
> >
> > Could we fix the indention here?

Can you give an example of how "fixed" looks like? This way is consistent with 
the comments in the main function.


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67270/#review203676
---


On May 23, 2018, 5:22 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67270/
> ---
> 
> (Updated May 23, 2018, 5:22 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The 'example.cpp' file in libprocess was missing the mandatory
> help string argument.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/examples/example.cpp 
> a97665ee6b75c5b31397b5a10002fa23a7fe370a 
> 
> 
> Diff: https://reviews.apache.org/r/67270/diff/1/
> 
> 
> Testing
> ---
> 
> Compiled the `example.cpp` file.
> 
> 
> Thanks,
> 
> Benno Evers
> 
>

Re: Review Request 67137: Avoided leaking file descriptors in Mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67137/
---

(Updated May 24, 2018, 1:13 p.m.)


Review request for mesos, Gilbert Song, Jie Yu, and James Peach.


Changes
---

Added a `TODO` as suggested by Jie.


Bugs: MESOS-8917
https://issues.apache.org/jira/browse/MESOS-8917


Repository: mesos


Description
---

Avoided leaking file descriptors in Mesos containerizer.


Diffs (updated)
-

  src/slave/containerizer/mesos/launch.cpp 
f25d90651ef32495c9161c3eaed8a327d1b2b926 


Diff: https://reviews.apache.org/r/67137/diff/8/

Changes: https://reviews.apache.org/r/67137/diff/7-8/


Testing
---

`make check`

I still need to confirm this patch in CI on a wider range of scenarios.


Thanks,

Benjamin Bannier

Review Request 67281: Reformated libprocess example code.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67281/
---

Review request for mesos and Alexander Rukletsov.


Repository: mesos


Description
---

Reformated libprocess example code.


Diffs
-

  3rdparty/libprocess/examples/example.cpp 
a97665ee6b75c5b31397b5a10002fa23a7fe370a 


Diff: https://reviews.apache.org/r/67281/diff/1/


Testing
---

Code reformatted with `clang-format` in Mesos style with manual tweaks for 
comments and single expression function bodies.


Thanks,

Benjamin Bannier

Review Request 67280: Added libprocess example code to build.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67280/
---

Review request for mesos and Benno Evers.


Repository: mesos


Description
---

Added libprocess example code to build.


Diffs
-

  3rdparty/libprocess/CMakeLists.txt 38522b1091a99141f75eccd7b54152bb40d00939 
  3rdparty/libprocess/Makefile.am b1d77f3a7ab1426d873d1f1d886e3c255ce32379 
  3rdparty/libprocess/examples/CMakeLists.txt PRE-CREATION 


Diff: https://reviews.apache.org/r/67280/diff/1/


Testing
---

`make check` (autotools, cmake)


Thanks,

Benjamin Bannier

Re: Review Request 67270: Fixed compilation error in libprocess example.



> On May 23, 2018, 8:39 p.m., Benjamin Bannier wrote:
> > Could you follow this up with a patch which compiles the example at least 
> > as part of `check`?
> > 
> > For cmake we could e.g., add a `CMakeLists.txt` to the `example/` directory 
> > with just
> > 
> > add_executable(example EXCLUDE_FROM_ALL example.cpp)
> > target_link_libraries(example process)
> > add_dependencies(libprocess-tests example)
> > 
> > and add the subdirectory to `libprocess/CMakeLists.txt`.
> > 
> > Similar for autotools.
> > 
> > Maybe we also should format the file according to our style guide in 
> > another patch.
> 
> Benno Evers wrote:
> Sorry, I don't really have the time right now to work on that. But I'll 
> volunteer to review the patch if you want to write it.

https://reviews.apache.org/r/67281/ for formatting of the current code, 
https://reviews.apache.org/r/67280/ for adding the fixed files to the build 
setups.


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67270/#review203676
---


On May 23, 2018, 7:22 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67270/
> ---
> 
> (Updated May 23, 2018, 7:22 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The 'example.cpp' file in libprocess was missing the mandatory
> help string argument.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/examples/example.cpp 
> a97665ee6b75c5b31397b5a10002fa23a7fe370a 
> 
> 
> Diff: https://reviews.apache.org/r/67270/diff/1/
> 
> 
> Testing
> ---
> 
> Compiled the `example.cpp` file.
> 
> 
> Thanks,
> 
> Benno Evers
> 
>

Re: Review Request 67281: Reformated libprocess example code.

2018-05-24 Thread Till Toenshoff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67281/#review203758
---


Ship it!




Ship It!

- Till Toenshoff


On May 24, 2018, 11:50 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67281/
> ---
> 
> (Updated May 24, 2018, 11:50 a.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Reformated libprocess example code.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/examples/example.cpp 
> a97665ee6b75c5b31397b5a10002fa23a7fe370a 
> 
> 
> Diff: https://reviews.apache.org/r/67281/diff/1/
> 
> 
> Testing
> ---
> 
> Code reformatted with `clang-format` in Mesos style with manual tweaks for 
> comments and single expression function bodies.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67270: Fixed various issues in libprocess example.

2018-05-24 Thread Benno Evers


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67270/
---

(Updated May 24, 2018, 12:38 p.m.)


Review request for mesos and Benjamin Mahler.


Changes
---

Fix various other issues.


Summary (updated)
-

Fixed various issues in libprocess example.


Repository: mesos


Description (updated)
---

* Added the mandatory help string argument to calls
  to 'route()', failing compilation.

* The response returned from the '/vars' endpoint had
  its 'type' field set to 'NONE', causing indefinite
  hanging when trying to access the endpoint. Switched
  to a constructor for OK that implicitly sets that field.

* Removed a stray pid variable in main.


Diffs (updated)
-

  3rdparty/libprocess/examples/example.cpp 
a97665ee6b75c5b31397b5a10002fa23a7fe370a 


Diff: https://reviews.apache.org/r/67270/diff/2/

Changes: https://reviews.apache.org/r/67270/diff/1-2/


Testing
---

Compiled the `example.cpp` file.


Thanks,

Benno Evers

Re: Review Request 67137: Avoided leaking file descriptors in Mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67137/#review203760
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67137']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67137

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67137/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (110 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1011 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (33 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (38 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (749 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (773 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (730 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (755 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (458628 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67137/logs/mesos-tests-stderr.log):

```
I0524 12:51:37.738133 18444 master.cpp:10843] Updating the state of task 
fec4d662-dbde-4a9d-806f-1d874cba27b5 of framework 
665c8921-e1fd-47fd-8df4-4684561c6b32- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 12:51:37.738133 20348 slave.cpp:3935] Shutting down framework 
665c8921-e1fd-47fd-8df4-4684561c6b32-
I0524 12:51:37.738133 20348 slave.cpp:6656] Shutting down executor 
'fec4d662-dbde-4a9d-806f-1d874cba27b5' of framework 
665c8921-e1fd-47fd-8df4-4684561c6b32- at executor(1)@192.10.1.5:51222
I0524 12:51:37.740131 20348 slave.cpp:929] Agent terminating
W0524 12:51:37.740131 20348 slave.cpp:3931]I0524 12:51:37.562160 15080 
exec.cpp:162] Version: 1.7.0
I0524 12:51:37.588127 16544 exec.cpp:236] Executor registered on agent 
665c8921-e1fd-47fd-8df4-4684561c6b32-S0
I0524 12:51:37.592159 18800 executor.cpp:178] Received SUBSCRIBED event
I0524 12:51:37.597162 18800 executor.cpp:182] Subscribed executor on 
windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 12:51:37.598161 18800 executor.cpp:178] Received LAUNCH event
I0524 12:51:37.602129 18800 executor.cpp:665] Starting task 
fec4d662-dbde-4a9d-806f-1d874cba27b5
I0524 12:51:37.693156 18800 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 12:51:37.711122 18800 executor.cpp:678] Forked command at 19000
I0524 12:51:37.740131 21464 exec.cpp:445] Executor asked to shutdown
I0524 12:51:37.741142 20788 executor.cpp:178] Received SHUTDOWN event
I0524 12:51:37.741142 20788 executor.cpp:781] Shutting down
I0524 12:51:37.741142 20788 executor.cpp:894] Sending SIGTERM to process tree 
at pid 19 Ignoring shutdown framework 665c8921-e1fd-47fd-8df4-4684561c6b32- 
because it is terminating
I0524 12:51:37.741142 18444 master.cpp:10942] Removing task 
fec4d662-dbde-4a9d-806f-1d874cba27b5 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 665c8921-e1fd-47fd-8df4-4684561c6b32- on 
agent 665c8921-e1fd-47fd-8df4-4684561c6b32-S0 at slave(448)@192.10.1.5:51201 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 12:51:37.744154 18444 master.cpp:1293] Agent 
665c8921-e1fd-47fd-8df4-4684561c6b32-S0 at slave(448)@192.10.1.5:51201 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 12:51:37.744154 18444 master.cpp:3303] Disconnecting agent 
665c8921-e1fd-47fd-8df4-4684561c6b32-S0 at slave(448)@192.10.1.5:51201 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 12:51:37.745160 18444 master.cpp:3322] Deactivating agent 
665c8921-e1fd-47fd-8df4-4684561c6b32-S0 at slave(448)@192.10.1.5:51201 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 12:51:37.745160 17880

Re: Review Request 67247: Added support script to check if Python >= 3.6 is available.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67247/
---

(Updated mai 24, 2018, 12:56 après-midi)


Review request for mesos and Andrew Schwartzmeyer.


Changes
---

Fixed script for Windows.


Bugs: MESOS-8770
https://issues.apache.org/jira/browse/MESOS-8770


Repository: mesos


Description
---

Added support script to check if Python >= 3.6 is available.


Diffs (updated)
-

  support/check-python3.py PRE-CREATION 


Diff: https://reviews.apache.org/r/67247/diff/2/

Changes: https://reviews.apache.org/r/67247/diff/1-2/


Testing
---

On my machine (macOS High Sierra), python is 2.7 and python3 is 3.6.
```
$ python check-python3.py
$ python3 check-python3.py
```

Then changed the loop in the script to have `for x in range(0, 7):`.

```
$ python check-python3.py
The support scripts will be upgraded to Python 3 by July 1st.
Make sure to install Python 3.6 on your machine before.
$ python3 check-python3.py
```
Then changed the script to have `elif sys.version_info[1] < 7:`
```
$ python check-python3.py
The support scripts will be upgraded to Python 3 by July 1st.
Make sure to install Python 3.6 on your machine before.
$ python3 check-python3.py
The support scripts will be upgraded to Python 3 by July 1st.
Make sure to install Python 3.6 on your machine before.
```


Thanks,

Armand Grillet

Re: Review Request 67247: Added support script to check if Python >= 3.6 is available.



> On mai 23, 2018, 7:47 après-midi, Andrew Schwartzmeyer wrote:
> > support/check-python3.py
> > Lines 34-37 (patched)
> > 
> >
> > While `which` isn't on Windows, I'm not opening an issue as I think 
> > this is adequate. The Windows users will still get the warning (no matter 
> > what), so I'm fine with this.

Fixed in the latest diff, I have tested the code using a PC running on Windows 
10.


- Armand


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67247/#review203682
---


On mai 22, 2018, 12:39 après-midi, Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67247/
> ---
> 
> (Updated mai 22, 2018, 12:39 après-midi)
> 
> 
> Review request for mesos and Andrew Schwartzmeyer.
> 
> 
> Bugs: MESOS-8770
> https://issues.apache.org/jira/browse/MESOS-8770
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added support script to check if Python >= 3.6 is available.
> 
> 
> Diffs
> -
> 
>   support/check-python3.py PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67247/diff/1/
> 
> 
> Testing
> ---
> 
> On my machine (macOS High Sierra), python is 2.7 and python3 is 3.6.
> ```
> $ python check-python3.py
> $ python3 check-python3.py
> ```
> 
> Then changed the loop in the script to have `for x in range(0, 7):`.
> 
> ```
> $ python check-python3.py
> The support scripts will be upgraded to Python 3 by July 1st.
> Make sure to install Python 3.6 on your machine before.
> $ python3 check-python3.py
> ```
> Then changed the script to have `elif sys.version_info[1] < 7:`
> ```
> $ python check-python3.py
> The support scripts will be upgraded to Python 3 by July 1st.
> Make sure to install Python 3.6 on your machine before.
> $ python3 check-python3.py
> The support scripts will be upgraded to Python 3 by July 1st.
> Make sure to install Python 3.6 on your machine before.
> ```
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>

Re: Review Request 67270: Fixed compilation issues in libprocess example.

2018-05-24 Thread Benno Evers


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67270/
---

(Updated May 24, 2018, 1:33 p.m.)


Review request for mesos and Benjamin Mahler.


Changes
---

Fix uncompilable code in commented-out sections.


Summary (updated)
-

Fixed compilation issues in libprocess example.


Repository: mesos


Description (updated)
---

* Added the mandatory help string argument to calls
  to 'route()', failing compilation.

* The response returned from the '/vars' endpoint had
  its 'type' field set to 'NONE', causing indefinite
  hanging when trying to access the endpoint. Switched
  to a constructor for OK that implicitly sets that field.

* Removed a stray pid variable in main.

* Added http:: namespace to disambiguate calls to 'post()'.

* Commented out example code that commits instant
  self-termination by default.


Diffs (updated)
-

  3rdparty/libprocess/examples/example.cpp 
8c507aefa050543dfa6184626c494918c96d35e7 


Diff: https://reviews.apache.org/r/67270/diff/4/

Changes: https://reviews.apache.org/r/67270/diff/3-4/


Testing
---

Compiled the `example.cpp` file.


Thanks,

Benno Evers

Re: Review Request 67280: Added libprocess example code to build.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67280/#review203765
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67270', '67280']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67280

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67280/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (123 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1073 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (39 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (72 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (714 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (738 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (730 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (756 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (457242 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67280/logs/mesos-tests-stderr.log):

```
I0524 14:26:57.811573  1096 master.cpp:10843] Updating the state of task 
59f01b76-34df-449c-b7c6-9dbb15562833 of framework 
f6480039-b1ed-4a2e-8d07-af9102b4b545- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 14:26:57.811573 22024 slave.cpp:3935] Shutting down framework 
f6480039-b1ed-4a2e-8d07-af9102b4b545-
I0524 14:26:57.811573 22024 slave.cpp:6656] Shutting down executor 
'59f01b76-34df-449c-b7c6-9dbbI0524 14:26:57.638581 21020 exec.cpp:162] Version: 
1.7.0
I0524 14:26:57.666584 16300 exec.cpp:236] Executor registered on agent 
f6480039-b1ed-4a2e-8d07-af9102b4b545-S0
I0524 14:26:57.671579  7096 executor.cpp:178] Received SUBSCRIBED event
I0524 14:26:57.676570  7096 executor.cpp:182] Subscribed executor on 
windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 14:26:57.676570  7096 executor.cpp:178] Received LAUNCH event
I0524 14:26:57.682579  7096 executor.cpp:665] Starting task 
59f01b76-34df-449c-b7c6-9dbb15562833
I0524 14:26:57.765583  7096 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 14:26:57.782577  7096 executor.cpp:678] Forked command at 16252
I0524 14:26:57.813585  6964 exec.cpp:445] Executor asked to shutdown
I0524 14:26:57.814581 20644 executor.cpp:178] Received SHUTDOWN event
I0524 14:26:57.814581 20644 executor.cpp:781] Shutting down
I0524 14:26:57.814581 20644 executor.cpp:894] Sending SIGTERM to process tree 
at pid 1615562833' of framework f6480039-b1ed-4a2e-8d07-af9102b4b545- at 
executor(1)@192.10.1.5:56055
I0524 14:26:57.813585 22024 slave.cpp:929] Agent terminating
W0524 14:26:57.813585 22024 slave.cpp:3931] Ignoring shutdown framework 
f6480039-b1ed-4a2e-8d07-af9102b4b545- because it is terminating
I0524 14:26:57.813585  1096 master.cpp:10942] Removing task 
59f01b76-34df-449c-b7c6-9dbb15562833 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework f6480039-b1ed-4a2e-8d07-af9102b4b545- on 
agent f6480039-b1ed-4a2e-8d07-af9102b4b545-S0 at slave(448)@192.10.1.5:56034 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 14:26:57.816574 18552 master.cpp:1293] Agent 
f6480039-b1ed-4a2e-8d07-af9102b4b545-S0 at slave(448)@192.10.1.5:56034 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 14:26:57.816574 18552 master.cpp:3303] Disconnecting agent 
f6480039-b1ed-4a2e-8d07-af9102b4b545-S0 at slave(448)@192.10.1.5:56034 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 14:26:57.817585 17128 hierarchical.cpp:344] Removed framework 
f6480039-b1ed-4a2e-8d07-af9102b4b545-
I0524 14:26:57.817585 18552 master.cpp:3322] Deactivating agent 
f6480039-b1ed-4a2e-8d07-af9102b4b545-S0 at sl

Review Request 67282: Added python3 to list of Pylint excluded files.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67282/
---

Review request for mesos and Andrew Schwartzmeyer.


Bugs: MESOS-8770
https://issues.apache.org/jira/browse/MESOS-8770


Repository: mesos


Description
---

This change ensures that pylint will not try to lint the new Python 3
support scripts if it is not run with Python 3. Having such a situation
results in unexpected errors such as "Unnecessary parens after 'print'
keyword". This change will not be applied in the Python 3 mesos-style.


Diffs
-

  support/mesos-style.py b7ee699e3898d795bfac97e88ad8e111392c0e3e 


Diff: https://reviews.apache.org/r/67282/diff/1/


Testing
---

Applied the entire chain, run some `git commit --amend` with and without 
`export MESOSSUPPORTPYTHON=3` (rebuilding the virtualenv was also done 
following this change) to see if the correct Python files were linted.


Thanks,

Armand Grillet

Re: Review Request 67059: Ported all support scripts to Python 3.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67059/
---

(Updated May 24, 2018, 2:45 p.m.)


Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Eric Chung, 
and Kevin Klues.


Changes
---

Changed dependency, improved README, and better pylint handling.


Bugs: MESOS-8770
https://issues.apache.org/jira/browse/MESOS-8770


Repository: mesos


Description
---

The scripts are in a temporary directory, support/python3.

The scripts have been ported using 2to3, the official tool to do so.
Many of these scripts require testing from the community before being
used by default.

The script building the virtual environment and the git hooks have
been updated to use the new scripts if the environment variable
`MESOSSUPPORTPYTHON` is set to `3` by the user.


Diffs (updated)
-

  support/README.md e812e9638ca555c56e4c52e52485240e75cb8229 
  support/build-virtualenv 850af89326721f34de20eb45a7e78fa391d031be 
  support/hooks/post-rewrite 1ab14abf711d1923a7ae69beb33581317009a94a 
  support/hooks/pre-commit 6faba98ab6db68aef1a54091a08b8db1eaac8701 
  support/python3/apply-reviews.py PRE-CREATION 
  support/python3/generate-endpoint-help.py PRE-CREATION 
  support/python3/jsonurl.py PRE-CREATION 
  support/python3/mesos-gtest-runner.py PRE-CREATION 
  support/python3/mesos-split.py PRE-CREATION 
  support/python3/mesos-style.py PRE-CREATION 
  support/python3/post-reviews.py PRE-CREATION 
  support/python3/push-commits.py PRE-CREATION 
  support/python3/test-upgrade.py PRE-CREATION 
  support/python3/verify-reviews.py PRE-CREATION 


Diff: https://reviews.apache.org/r/67059/diff/4/

Changes: https://reviews.apache.org/r/67059/diff/3-4/


Testing
---

All the files are OK for our linter, I have tested `mesos-style.py` and 
`post-reviews.py` (used for this review request).

We will likely see error messages `TypeError: cannot use a string pattern on a 
bytes-like object`, they are very easy to fix (we just need to add 
`.decode("utf-8")` but detecting all of them requires to use the scripts. 
Please do so and create issues.


Thanks,

Armand Grillet

Re: Review Request 67235: Added per-framework metrics for types of resources contained in offers.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67235/#review203768
---



Patch looks great!

Reviews applied: [66882, 66819, 66820, 66822, 66823, 66845, 66825, 66846, 
66847, 66841, 66842, 66843, 67147, 66844, 66855, 66861, 66856, 66870, 66874, 
67043, 66883, 67187, 67235]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 24, 2018, 12:14 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67235/
> ---
> 
> (Updated May 24, 2018, 12:14 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Gaston Kleiman, Gilbert Song, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8940
> https://issues.apache.org/jira/browse/MESOS-8940
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added per-framework metrics for types of resources contained in offers.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp 41862db9900acde85a62d2fea85459691c68556e 
>   src/master/metrics.hpp ec76dbcd1d1fa5349d62ce73fb9603e1986a776b 
>   src/master/metrics.cpp e46ead79f3f29e285426f9d061337077f453aa45 
>   src/tests/master_tests.cpp 6a0fa988aebaf1cf66abaaca0c628f981c6ccc31 
> 
> 
> Diff: https://reviews.apache.org/r/67235/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 67282: Added python3 to list of Pylint excluded files.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67282/#review203770
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67247', '67099', '67282']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67282

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67282/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (110 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (997 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (38 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (36 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (76 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (785 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (811 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (730 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (754 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (443620 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67282/logs/mesos-tests-stderr.log):

```
I0524 15:26:27.966481 15324 slave.cpp:3935] Shutting down framework 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-
I0524 15:26:27.966481 16308 master.cpp:10843] Updating the state of task 
667f58ea-9381-4ce8-bcdf-eddb3135a213 of framework 
5d809c77-cd24-4d64-8e7f-8b04ac697a50- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 15:26:27.9664I0524 15:26:27.801475 17168 exec.cpp:162] Version: 1.7.0
I0524 15:26:27.826457  9044 exec.cpp:236] Executor registered on agent 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-S0
I0524 15:26:27.830483  9168 executor.cpp:178] Received SUBSCRIBED event
I0524 15:26:27.835469  9168 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 15:26:27.835469  9168 executor.cpp:178] Received LAUNCH event
I0524 15:26:27.840473  9168 executor.cpp:665] Starting task 
667f58ea-9381-4ce8-bcdf-eddb3135a213
I0524 15:26:27.921468  9168 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 15:26:27.938468  9168 executor.cpp:678] Forked command at 8128
I0524 15:26:27.968438 19540 exec.cpp:445] Executor asked to shutdown
I0524 15:26:27.968438 12604 executor.cpp:178] Received SHUTDOWN event
I0524 15:26:27.969439 12604 executor.cpp:781] Shutting down
I0524 15:26:27.969439 12604 executor.cpp:894] Sending SIGTERM to process tree 
at pid 81281 15324 slave.cpp:6656] Shutting down executor 
'667f58ea-9381-4ce8-bcdf-eddb3135a213' of framework 
5d809c77-cd24-4d64-8e7f-8b04ac697a50- at executor(1)@192.10.1.6:63075
I0524 15:26:27.968438 15324 slave.cpp:929] Agent terminating
W0524 15:26:27.968438 15324 slave.cpp:3931] Ignoring shutdown framework 
5d809c77-cd24-4d64-8e7f-8b04ac697a50- because it is terminating
I0524 15:26:27.968438 16308 master.cpp:10942] Removing task 
667f58ea-9381-4ce8-bcdf-eddb3135a213 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 5d809c77-cd24-4d64-8e7f-8b04ac697a50- on 
agent 5d809c77-cd24-4d64-8e7f-8b04ac697a50-S0 at slave(448)@192.10.1.6:63054 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 15:26:27.971441 16308 master.cpp:1293] Agent 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-S0 at slave(448)@192.10.1.6:63054 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 15:26:27.972435 16308 master.cpp:3303] Disconnecting agent 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-S0 at slave(448)@192.10.1.6:63054 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 15:26:27.972435 19980 hierarchical.cpp:344] Removed framework 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-
I0524 15:26:27.972435 16308 master.cpp:3322] Deactivating agent 
5d809c77-cd24-4d64-8e7f-8b04ac697a50-S

Review Request 67285: Moved JWT test keys into the tests source file.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67285/
---

Review request for mesos, Andrew Schwartzmeyer and Till Toenshoff.


Repository: mesos


Description
---

The JWT test keys were only used by the JWT tests, so there was
no necessity for them to be in a header file. Moved them into the
source file to fix `make distcheck` which was failing because the
`jwt_keys.hpp` header wasn't included in the distribution.


Diffs
-

  3rdparty/libprocess/src/tests/jwt_keys.hpp 
0fb9a989da4d0ff2eaf6b304922ae68206319079 
  3rdparty/libprocess/src/tests/jwt_tests.cpp 
581438d7b8811474d694e9482dfffe8b5068e53f 


Diff: https://reviews.apache.org/r/67285/diff/1/


Testing
---

make check (Fedora 27)


Thanks,

James Peach

Re: Review Request 67241: Added isolator logs for volume/secret isolator and container logger.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67241/#review203773
---



Bad patch!

Reviews applied: [67241]

Failed command: python support/apply-reviews.py -n -r 67241

Error:
2018-05-24 15:57:33 URL:https://reviews.apache.org/r/67241/diff/raw/ 
[2212/2212] -> "67241.patch" [1]
error: patch failed: src/slave/containerizer/mesos/io/switchboard.cpp:302
error: src/slave/containerizer/mesos/io/switchboard.cpp: patch does not apply
error: patch failed: 
src/slave/containerizer/mesos/isolators/volume/secret.cpp:291
error: src/slave/containerizer/mesos/isolators/volume/secret.cpp: patch does 
not apply

Full log: https://builds.apache.org/job/Mesos-Reviewbot/22534/console

- Mesos Reviewbot


On May 24, 2018, 1:08 a.m., Gilbert Song wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67241/
> ---
> 
> (Updated May 24, 2018, 1:08 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Jie Yu, Qian Zhang, and Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added isolator logs for volume/secret isolator and container logger.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/io/switchboard.cpp 
> 6d99fb54a707d18464bbfd57b5021da9ccd66c57 
>   src/slave/containerizer/mesos/isolators/volume/secret.cpp 
> 8071e4ee808bc825b13a6291767778d6ce3c2746 
> 
> 
> Diff: https://reviews.apache.org/r/67241/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gilbert Song
> 
>

Re: Review Request 67271: Change ambiguous parameter name.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67271/#review203774
---


Ship it!




Ship It!

- James Peach


On May 23, 2018, 6:42 p.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67271/
> ---
> 
> (Updated May 23, 2018, 6:42 p.m.)
> 
> 
> Review request for mesos and James Peach.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Change ambiguous parameter name.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
> 
> 
> Diff: https://reviews.apache.org/r/67271/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67059: Ported all support scripts to Python 3.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67059/#review203775
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67247', '67099', '67282', '67059']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67059

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67059/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (105 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1000 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (35 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (769 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (793 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (734 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (759 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (440516 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67059/logs/mesos-tests-stderr.log):

```
I0524 16:12:37.297600 21032 master.cpp:10843] Updating the state of task 
1d5d0213-21ca-4cfe-994d-eb62f07f5648 oI0524 16:12:37.140612 19832 exec.cpp:162] 
Version: 1.7.0
I0524 16:12:37.165616 22184 exec.cpp:236] Executor registered on agent 
cef0344d-7a02-417f-aadb-7eb5013cc9b4-S0
I0524 16:12:37.168619 18068 executor.cpp:178] Received SUBSCRIBED event
I0524 16:12:37.173619 18068 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 16:12:37.173619 18068 executor.cpp:178] Received LAUNCH event
I0524 16:12:37.178612 18068 executor.cpp:665] Starting task 
1d5d0213-21ca-4cfe-994d-eb62f07f5648
I0524 16:12:37.256613 18068 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 16:12:37.272620 18068 executor.cpp:678] Forked command at 16964
I0524 16:12:37.299602 18816 exec.cpp:445] Executor asked to shutdown
I0524 16:12:37.300601  8596 executor.cpp:178] Received SHUTDOWN event
I0524 16:12:37.300601  8596 executor.cpp:781] Shutting down
I0524 16:12:37.300601  8596 executor.cpp:894] Sending SIGTERM to process tree 
at pid 16f framework cef0344d-7a02-417f-aadb-7eb5013cc9b4- (latest state: 
TASK_KILLED, status update state: TASK_KILLED)
I0524 16:12:37.297600 16972 slave.cpp:3935] Shutting down framework 
cef0344d-7a02-417f-aadb-7eb5013cc9b4-
I0524 16:12:37.298601 16972 slave.cpp:6656] Shutting down executor 
'1d5d0213-21ca-4cfe-994d-eb62f07f5648' of framework 
cef0344d-7a02-417f-aadb-7eb5013cc9b4- at executor(1)@192.10.1.6:49351
I0524 16:12:37.299602 16972 slave.cpp:929] Agent terminating
W0524 16:12:37.300601 16972 slave.cpp:3931] Ignoring shutdown framework 
cef0344d-7a02-417f-aadb-7eb5013cc9b4- because it is terminating
I0524 16:12:37.300601 21032 master.cpp:10942] Removing task 
1d5d0213-21ca-4cfe-994d-eb62f07f5648 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework cef0344d-7a02-417f-aadb-7eb5013cc9b4- on 
agent cef0344d-7a02-417f-aadb-7eb5013cc9b4-S0 at slave(448)@192.10.1.6:49330 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 16:12:37.304608 21032 master.cpp:1293] Agent 
cef0344d-7a02-417f-aadb-7eb5013cc9b4-S0 at slave(448)@192.10.1.6:49330 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 16:12:37.304608 21032 master.cpp:3303] Disconnecting agent 
cef0344d-7a02-417f-aadb-7eb5013cc9b4-S0 at slave(448)@192.10.1.6:49330 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 16:12:37.304608 21032 master.cpp:3322] Deactivating agent 
cef0344d-7a02-417f-aadb-7eb5013cc9b4-S0 at slave(448)@192.10.1.6:49330 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0

Re: Review Request 67285: Moved JWT test keys into the tests source file.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67285/#review203776
---


Fix it, then Ship it!




Oh it was `make dist` that was failing CI. Nice find.


3rdparty/libprocess/src/tests/jwt_tests.cpp
Line 26 (original)


I think this file is also listed in a Makefile somewhere and that should 
get deleted...


- Andrew Schwartzmeyer


On May 24, 2018, 8:48 a.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67285/
> ---
> 
> (Updated May 24, 2018, 8:48 a.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and Till Toenshoff.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The JWT test keys were only used by the JWT tests, so there was
> no necessity for them to be in a header file. Moved them into the
> source file to fix `make distcheck` which was failing because the
> `jwt_keys.hpp` header wasn't included in the distribution.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/src/tests/jwt_keys.hpp 
> 0fb9a989da4d0ff2eaf6b304922ae68206319079 
>   3rdparty/libprocess/src/tests/jwt_tests.cpp 
> 581438d7b8811474d694e9482dfffe8b5068e53f 
> 
> 
> Diff: https://reviews.apache.org/r/67285/diff/1/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 67285: Moved JWT test keys into the tests source file.



> On May 24, 2018, 4:15 p.m., Andrew Schwartzmeyer wrote:
> > 3rdparty/libprocess/src/tests/jwt_tests.cpp
> > Line 26 (original)
> > 
> >
> > I think this file is also listed in a Makefile somewhere and that 
> > should get deleted...

No, it's not listed.
```
[jpeach@jpeach asf-mesos]$ git grep jwt_keys.hpp
[jpeach@jpeach asf-mesos]$
```


- James


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67285/#review203776
---


On May 24, 2018, 3:48 p.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67285/
> ---
> 
> (Updated May 24, 2018, 3:48 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and Till Toenshoff.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The JWT test keys were only used by the JWT tests, so there was
> no necessity for them to be in a header file. Moved them into the
> source file to fix `make distcheck` which was failing because the
> `jwt_keys.hpp` header wasn't included in the distribution.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/src/tests/jwt_keys.hpp 
> 0fb9a989da4d0ff2eaf6b304922ae68206319079 
>   3rdparty/libprocess/src/tests/jwt_tests.cpp 
> 581438d7b8811474d694e9482dfffe8b5068e53f 
> 
> 
> Diff: https://reviews.apache.org/r/67285/diff/1/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 66668: Removed `destroyed` from `Container` struct in composing containerizer.

2018-05-24 Thread Greg Mann


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8/#review203778
---


Ship it!




Ship It!

- Greg Mann


On April 17, 2018, 3:23 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8/
> ---
> 
> (Updated April 17, 2018, 3:23 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8712
> https://issues.apache.org/jira/browse/MESOS-8712
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously, we stored `destroyed` promise for each container and used
> it to guarantee that `destroy()` returns a non-empty value when the
> destroy-in-progress stops an launch-in-progress using the next
> containerizer. Since `wait()` and `destroy()` return the same
> `ContainerTermination` value when called with the same ContainerID
> argument, we can remove `destroyed` promise and add callbacks to
> clean up `containers_` map instead.
> 
> Moreover, we added a clean up for terminated containers that have
> been recovered after agent's restart.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/composing.cpp 
> 1fb79f53b48154ecbd3b0165b6a8002c871e6dce 
> 
> 
> Diff: https://reviews.apache.org/r/8/diff/9/
> 
> 
> Testing
> ---
> 
> internal CI
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 67130: Updated comments related to `wait`, `destroy` containerizer methods.

2018-05-24 Thread Greg Mann


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67130/#review203780
---


Ship it!




Ship It!

- Greg Mann


On May 15, 2018, 1:50 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67130/
> ---
> 
> (Updated May 15, 2018, 1:50 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Greg Mann, Jie Yu, and Qian 
> Zhang.
> 
> 
> Bugs: MESOS-8740
> https://issues.apache.org/jira/browse/MESOS-8740
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch updates description of `wait()` and `destroy()` methods
> of the containerizer API.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/containerizer.hpp 
> 5817b1dfacd02d818d867ddf214ba316182ebedc 
> 
> 
> Diff: https://reviews.apache.org/r/67130/diff/4/
> 
> 
> Testing
> ---
> 
> None: not a functional change.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 67282: Added python3 to list of Pylint excluded files.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67282/#review203784
---


Ship it!




Ship It!

- Andrew Schwartzmeyer


On May 24, 2018, 7:34 a.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67282/
> ---
> 
> (Updated May 24, 2018, 7:34 a.m.)
> 
> 
> Review request for mesos and Andrew Schwartzmeyer.
> 
> 
> Bugs: MESOS-8770
> https://issues.apache.org/jira/browse/MESOS-8770
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This change ensures that pylint will not try to lint the new Python 3
> support scripts if it is not run with Python 3. Having such a situation
> results in unexpected errors such as "Unnecessary parens after 'print'
> keyword". This change will not be applied in the Python 3 mesos-style.
> 
> 
> Diffs
> -
> 
>   support/mesos-style.py b7ee699e3898d795bfac97e88ad8e111392c0e3e 
> 
> 
> Diff: https://reviews.apache.org/r/67282/diff/1/
> 
> 
> Testing
> ---
> 
> Applied the entire chain, run some `git commit --amend` with and without 
> `export MESOSSUPPORTPYTHON=3` (rebuilding the virtualenv was also done 
> following this change) to see if the correct Python files were linted.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>

Re: Review Request 67285: Moved JWT test keys into the tests source file.



> On May 24, 2018, 9:15 a.m., Andrew Schwartzmeyer wrote:
> > 3rdparty/libprocess/src/tests/jwt_tests.cpp
> > Line 26 (original)
> > 
> >
> > I think this file is also listed in a Makefile somewhere and that 
> > should get deleted...
> 
> James Peach wrote:
> No, it's not listed.
> ```
> [jpeach@jpeach asf-mesos]$ git grep jwt_keys.hpp
> [jpeach@jpeach asf-mesos]$
> ```

That's I get for reviewing from my cell phone... ;)


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67285/#review203776
---


On May 24, 2018, 8:48 a.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67285/
> ---
> 
> (Updated May 24, 2018, 8:48 a.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and Till Toenshoff.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The JWT test keys were only used by the JWT tests, so there was
> no necessity for them to be in a header file. Moved them into the
> source file to fix `make distcheck` which was failing because the
> `jwt_keys.hpp` header wasn't included in the distribution.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/src/tests/jwt_keys.hpp 
> 0fb9a989da4d0ff2eaf6b304922ae68206319079 
>   3rdparty/libprocess/src/tests/jwt_tests.cpp 
> 581438d7b8811474d694e9482dfffe8b5068e53f 
> 
> 
> Diff: https://reviews.apache.org/r/67285/diff/1/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 67285: Moved JWT test keys into the tests source file.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67285/#review203786
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67285']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (104 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (997 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (33 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (36 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (71 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (690 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (714 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (733 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (756 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (437195 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285/logs/mesos-tests-stderr.log):

```
I0524 16:58:45.636476 18624 exec.cpp:236] Executor registered on agent 
63433979-6344-4f2e-a387-292b1267aa98-S0
I0524 16:58:45.640498 12736 executor.cpp:178] Received SUBSCRIBED event
I0524 16:58:45.644471 12736 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 16:58:45.645474 12736 executor.cpp:178] Received LAUNCH event
I0524 16:58:45.649478 12736 executor.cpp:665] Starting task 
3e38c67d-12fe-44f6-a749-d32af9b01b5c
I0524 16:58:45.727474 12736 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 16:58:45.77 12736 executor.cpp:678] Forked command at 11848
I0524 16:58:45.773442 11528 exec.cpp:445] Executor asked to shutdown
I0524 16:58:45.77 16156 executor.cpp:178] Received SHUTDOWN event
I0524 16:58:45.77 16156 executor.cpp:781] Shutting down
I0524 16:58:45.77 16156 executor.cpp:894] Sending SIGTERM to process tree 
at pid 11al.cpp:405] Deactivated framework 
63433979-6344-4f2e-a387-292b1267aa98-
I0524 16:58:45.771445 19028 slave.cpp:3935] Shutting down framework 
63433979-6344-4f2e-a387-292b1267aa98-
I0524 16:58:45.771445 20984 master.cpp:10843] Updating the state of task 
3e38c67d-12fe-44f6-a749-d32af9b01b5c of framework 
63433979-6344-4f2e-a387-292b1267aa98- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 16:58:45.772445 19028 slave.cpp:6656] Shutting down executor 
'3e38c67d-12fe-44f6-a749-d32af9b01b5c' of framework 
63433979-6344-4f2e-a387-292b1267aa98- at executor(1)@192.10.1.6:52010
I0524 16:58:45.773442 19028 slave.cpp:929] Agent terminating
W0524 16:58:45.77 19028 slave.cpp:3931] Ignoring shutdown framework 
63433979-6344-4f2e-a387-292b1267aa98- because it is terminating
I0524 16:58:45.77 20984 master.cpp:10942] Removing task 
3e38c67d-12fe-44f6-a749-d32af9b01b5c with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 63433979-6344-4f2e-a387-292b1267aa98- on 
agent 63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 16:58:45.778443 20984 master.cpp:1293] Agent 
63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 16:58:45.778443 20984 master.cpp:3303] Disconnecting agent 
63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 16:58:45.778443 20984 master.cpp:3322] Deactivating agent 
63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 16:5

Review Request 67289: Introduced an overload for os::environment.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67289/
---

Review request for mesos, Andrew Schwartzmeyer and James Peach.


Repository: mesos


Description
---

This overload allows us to get a specific environment variable.


Diffs
-

  3rdparty/stout/include/stout/os/environment.hpp 
17a929cf9516413af1cbb64fb5e3d1eaf80b217c 


Diff: https://reviews.apache.org/r/67289/diff/1/


Testing
---

make check


Thanks,

Jie Yu

Review Request 67290: Removed an unneeded test.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67290/
---

Review request for mesos and James Peach.


Repository: mesos


Description
---

This test is covered by many other tests, and is testing for a
deprecated feature. Thus remove this test.


Diffs
-

  src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 


Diff: https://reviews.apache.org/r/67290/diff/1/


Testing
---

make check


Thanks,

Jie Yu

Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/
---

Review request for mesos, Andrew Schwartzmeyer and James Peach.


Repository: mesos


Description
---

Used `$SUDO_USER` instead because `nobody` sometimes cannot access
direcotries under `$HOME` of the current user running the tests.


Diffs
-

  src/tests/container_logger_tests.cpp 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
  src/tests/containerizer/capabilities_tests.cpp 
734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
  src/tests/containerizer/cgroups_isolator_tests.cpp 
40c18a1528f2da32c608b6c774500933a08562ea 
  src/tests/containerizer/docker_containerizer_tests.cpp 
e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
  src/tests/containerizer/mesos_containerizer_tests.cpp 
01f2b38cfa67b144298c361e92170322864ac201 
  src/tests/containerizer/provisioner_docker_tests.cpp 
c664ff807583d587d94b0ab797330d5d3daf7657 
  src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
0400052a1c73ccc15c932c573cfb324ca83e2e59 
  src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
  src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
  src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 


Diff: https://reviews.apache.org/r/67291/diff/1/


Testing
---

sudo make check


Thanks,

Jie Yu

Re: Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203789
---




src/tests/slave_tests.cpp
Lines 1041-1049 (original), 1037-1045 (patched)


Forgot to remove this.


- Jie Yu


On May 24, 2018, 5:09 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 5:09 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Review Request 67288: White list fds that child processes can inherit in mesos containerizer.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/
---

Review request for mesos, Akash Gupta and Andrew Schwartzmeyer.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description
---

White list fds that child processes can inherit in mesos containerizer.


Diffs
-

  src/slave/containerizer/mesos/containerizer.cpp 
eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
  src/slave/containerizer/mesos/launcher.hpp 
f69d934d2e1a129e10df8c7f5c78723e832adc7d 
  src/slave/containerizer/mesos/launcher.cpp 
2fe47d368cb82a46328e1f636baa836272db244c 
  src/slave/containerizer/mesos/linux_launcher.hpp 
0ea9b875ae46cadea483bc8dd8bf4907fd324dc9 
  src/slave/containerizer/mesos/linux_launcher.cpp 
80e444501e429c1e1ae354abcd51f86430316ada 


Diff: https://reviews.apache.org/r/67288/diff/1/


Testing
---


Thanks,

Radhika Jandhyala

Review Request 67287: White list fds that child processes can inherit in libprocess.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67287/
---

Review request for mesos and Andrew Schwartzmeyer.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description
---

White list fds that child processes can inherit in libprocess.


Diffs
-

  3rdparty/libprocess/include/process/subprocess.hpp 
6a1262340c333b617402637e648c12769827ffc4 
  3rdparty/libprocess/src/subprocess.cpp 
d7a725363251f9c54072cd7551f5598696938308 
  3rdparty/libprocess/src/subprocess_windows.hpp 
c7ed0ad18f5b46a1d5ac2a6e51883aefb7c1692f 


Diff: https://reviews.apache.org/r/67287/diff/1/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Review Request 67286: White list fds that child processes can inherit in stout.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67286/
---

Review request for mesos, Akash Gupta and Andrew Schwartzmeyer.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description
---

White list fds that child processes can inherit in stout.


Diffs
-

  3rdparty/stout/include/stout/internal/windows/inherit.hpp 
7dbde820e775cbaeb8db4bc4559ab432903e75ea 
  3rdparty/stout/include/stout/os/windows/shell.hpp 
8da612af2888ff4d4d458ea5b16cdb08779b6f4c 


Diff: https://reviews.apache.org/r/67286/diff/1/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Re: Review Request 67285: Moved JWT test keys into the tests source file.



> On May 24, 2018, 9:59 a.m., Mesos Reviewbot Windows wrote:
> > FAIL: Some of the unit tests failed. Please check the relevant logs.
> > 
> > Reviews applied: `['67285']`
> > 
> > Failed command: `Start-MesosCITesting`
> > 
> > All the build artifacts available at: 
> > http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285
> > 
> > Relevant logs:
> > 
> > - 
> > [mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285/logs/mesos-tests-stdout.log):
> > 
> > ```
> > [   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (104 ms)
> > [--] 9 tests from Endpoint/SlaveEndpointTest (997 ms total)
> > 
> > [--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
> > [ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
> > [   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 
> > (33 ms)
> > [ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
> > [   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 
> > (36 ms)
> > [--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (71 
> > ms total)
> > 
> > [--] 1 test from IsolationFlag/CpuIsolatorTest
> > [ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
> > [   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (690 ms)
> > [--] 1 test from IsolationFlag/CpuIsolatorTest (714 ms total)
> > 
> > [--] 1 test from IsolationFlag/MemoryIsolatorTest
> > [ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
> > [   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (733 ms)
> > [--] 1 test from IsolationFlag/MemoryIsolatorTest (756 ms total)
> > 
> > [--] Global test environment tear-down
> > [==] 981 tests from 95 test cases ran. (437195 ms total)
> > [  PASSED  ] 980 tests.
> > [  FAILED  ] 1 test, listed below:
> > [  FAILED  ] 
> > DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange
> > 
> >  1 FAILED TEST
> >   YOU HAVE 220 DISABLED TESTS
> > 
> > ```
> > 
> > - 
> > [mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67285/logs/mesos-tests-stderr.log):
> > 
> > ```
> > I0524 16:58:45.636476 18624 exec.cpp:236] Executor registered on agent 
> > 63433979-6344-4f2e-a387-292b1267aa98-S0
> > I0524 16:58:45.640498 12736 executor.cpp:178] Received SUBSCRIBED event
> > I0524 16:58:45.644471 12736 executor.cpp:182] Subscribed executor on 
> > windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
> > I0524 16:58:45.645474 12736 executor.cpp:178] Received LAUNCH event
> > I0524 16:58:45.649478 12736 executor.cpp:665] Starting task 
> > 3e38c67d-12fe-44f6-a749-d32af9b01b5c
> > I0524 16:58:45.727474 12736 executor.cpp:485] Running 
> > 'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
> > I0524 16:58:45.77 12736 executor.cpp:678] Forked command at 11848
> > I0524 16:58:45.773442 11528 exec.cpp:445] Executor asked to shutdown
> > I0524 16:58:45.77 16156 executor.cpp:178] Received SHUTDOWN event
> > I0524 16:58:45.77 16156 executor.cpp:781] Shutting down
> > I0524 16:58:45.77 16156 executor.cpp:894] Sending SIGTERM to process 
> > tree at pid 11al.cpp:405] Deactivated framework 
> > 63433979-6344-4f2e-a387-292b1267aa98-
> > I0524 16:58:45.771445 19028 slave.cpp:3935] Shutting down framework 
> > 63433979-6344-4f2e-a387-292b1267aa98-
> > I0524 16:58:45.771445 20984 master.cpp:10843] Updating the state of task 
> > 3e38c67d-12fe-44f6-a749-d32af9b01b5c of framework 
> > 63433979-6344-4f2e-a387-292b1267aa98- (latest state: TASK_KILLED, 
> > status update state: TASK_KILLED)
> > I0524 16:58:45.772445 19028 slave.cpp:6656] Shutting down executor 
> > '3e38c67d-12fe-44f6-a749-d32af9b01b5c' of framework 
> > 63433979-6344-4f2e-a387-292b1267aa98- at executor(1)@192.10.1.6:52010
> > I0524 16:58:45.773442 19028 slave.cpp:929] Agent terminating
> > W0524 16:58:45.77 19028 slave.cpp:3931] Ignoring shutdown framework 
> > 63433979-6344-4f2e-a387-292b1267aa98- because it is terminating
> > I0524 16:58:45.77 20984 master.cpp:10942] Removing task 
> > 3e38c67d-12fe-44f6-a749-d32af9b01b5c with resources cpus(allocated: *):4; 
> > mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
> > *):[31000-32000] of framework 63433979-6344-4f2e-a387-292b1267aa98- on 
> > agent 63433979-6344-4f2e-a387-292b1267aa98-S0 at 
> > slave(448)@192.10.1.6:51989 
> > (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
> > I0524 16:58:45.778443 20984 master.cpp:1293] Agent 
> > 63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
> > (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) 
> > disconnected
> > I0524 16:58:45.778443 20984 master.cpp:3303] Disconnecting agent 
> > 63433979-6344-4f2e-a387-292b1267aa98-S0 at slave(448)@192.10.1.6:51989 
> > (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.clouda

Re: Review Request 67290: Removed an unneeded test.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67290/
---

(Updated May 24, 2018, 5:49 p.m.)


Review request for mesos and James Peach.


Changes
---

Rebased.


Repository: mesos


Description
---

This test is covered by many other tests, and is testing for a
deprecated feature. Thus remove this test.


Diffs (updated)
-

  src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 


Diff: https://reviews.apache.org/r/67290/diff/2/

Changes: https://reviews.apache.org/r/67290/diff/1-2/


Testing
---

make check


Thanks,

Jie Yu

Re: Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/
---

(Updated May 24, 2018, 5:49 p.m.)


Review request for mesos, Andrew Schwartzmeyer and James Peach.


Changes
---

Used os::getenv instead.


Repository: mesos


Description
---

Used `$SUDO_USER` instead because `nobody` sometimes cannot access
direcotries under `$HOME` of the current user running the tests.


Diffs (updated)
-

  src/tests/container_logger_tests.cpp 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
  src/tests/containerizer/capabilities_tests.cpp 
734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
  src/tests/containerizer/cgroups_isolator_tests.cpp 
40c18a1528f2da32c608b6c774500933a08562ea 
  src/tests/containerizer/docker_containerizer_tests.cpp 
e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
  src/tests/containerizer/mesos_containerizer_tests.cpp 
01f2b38cfa67b144298c361e92170322864ac201 
  src/tests/containerizer/provisioner_docker_tests.cpp 
c664ff807583d587d94b0ab797330d5d3daf7657 
  src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
0400052a1c73ccc15c932c573cfb324ca83e2e59 
  src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
  src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
  src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 


Diff: https://reviews.apache.org/r/67291/diff/2/

Changes: https://reviews.apache.org/r/67291/diff/1-2/


Testing
---

sudo make check


Thanks,

Jie Yu

Re: Review Request 67271: Change ambiguous parameter name.

2018-05-24 Thread Gilbert Song


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67271/#review203791
---


Fix it, then Ship it!




usually we start the commit msg with a transformed verb. (e.g., `Changed`). 
here I would sugggest something like `Renamed an ambiguous variable in network 
port isolator.`, which may be more intuitive.


src/slave/containerizer/mesos/isolators/network/ports.cpp
Lines 532-533 (patched)


fit in oneline?


- Gilbert Song


On May 23, 2018, 11:42 a.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67271/
> ---
> 
> (Updated May 23, 2018, 11:42 a.m.)
> 
> 
> Review request for mesos and James Peach.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Change ambiguous parameter name.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
> 
> 
> Diff: https://reviews.apache.org/r/67271/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67195: Adding enforce_container_ports flag for network ports isolation.

2018-05-24 Thread Gilbert Song


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/#review203792
---



we need to update `configure.md` correspondingly.

- Gilbert Song


On May 23, 2018, 11:33 a.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67195/
> ---
> 
> (Updated May 23, 2018, 11:33 a.m.)
> 
> 
> Review request for mesos, James Peach and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> To reduce deployment risk, a nonenforce mode is added for network
> port isolator. When this flag is set as false(default is false),
> even task uses ports not in the container resources, the container
> will not raise any limitation.
> 
> Add new test for this flag and update the existing tests
> 
> 
> Diffs
> -
> 
>   docs/isolators/network-ports.md ea63968481ce52c46e0a98e242da49baf6962009 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
>   src/slave/flags.hpp a839591a2b66444ad97fced0620201dde656352d 
>   src/slave/flags.cpp a319b5ea633c41fd8a252c5e1617ac52d1480ba5 
>   src/tests/containerizer/ports_isolator_tests.cpp 
> c5b9f926047792e7f9d1f0937fa5355b1dd77965 
> 
> 
> Diff: https://reviews.apache.org/r/67195/diff/3/
> 
> 
> Testing
> ---
> 
> New test added for the flag; Related unit tests passed.
> 
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_AllocatedPorts (1906 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortsResource (1788 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortEnforement (17001 ms)
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203794
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67289', '67290', '67291']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67291

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67291/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (107 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1016 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (34 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (38 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (740 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (765 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (733 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (757 ms total)

[--] Global test environment tear-down
[==] 980 tests from 95 test cases ran. (452607 ms total)
[  PASSED  ] 979 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67291/logs/mesos-tests-stderr.log):

```
I0524 17:58:28.605190 12804 master.cpp:10843] Updating the state of task 
13880c12-575d-42f2-8bfc-360e9549a694 of framework 
1cd8243a-73d8-454a-8360-9c407e93a3a3- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 17:58:28.605190  9228 slave.cpp:3935] Shutting down framework 
1cd8243a-73d8-454a-8360-9c407e93a3a3-
I0524 17I0524 17:58:28.442185 21024 exec.cpp:162] Version: 1.7.0
I0524 17:58:28.467173 13408 exec.cpp:236] Executor registered on agent 
1cd8243a-73d8-454a-8360-9c407e93a3a3-S0
I0524 17:58:28.471176 16896 executor.cpp:178] Received SUBSCRIBED event
I0524 17:58:28.475186 16896 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 17:58:28.475186 16896 executor.cpp:178] Received LAUNCH event
I0524 17:58:28.480186 16896 executor.cpp:665] Starting task 
13880c12-575d-42f2-8bfc-360e9549a694
I0524 17:58:28.559188 16896 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 17:58:28.577188 16896 executor.cpp:678] Forked command at 16180
I0524 17:58:28.608178 21860 exec.cpp:445] Executor asked to shutdown
I0524 17:58:28.608178 18556 executor.cpp:178] Received SHUTDOWN event
I0524 17:58:28.608178 18556 executor.cpp:781] Shutting down
I0524 17:58:28.609171 18556 executor.cpp:894] Sending SIGTERM to process tree 
at pid 16:58:28.606176  9228 slave.cpp:6656] Shutting down executor 
'13880c12-575d-42f2-8bfc-360e9549a694' of framework 
1cd8243a-73d8-454a-8360-9c407e93a3a3- at executor(1)@192.10.1.6:54636
I0524 17:58:28.608178  9228 slave.cpp:929] Agent terminating
W0524 17:58:28.608178  9228 slave.cpp:3931] Ignoring shutdown framework 
1cd8243a-73d8-454a-8360-9c407e93a3a3- because it is terminating
I0524 17:58:28.608178 12804 master.cpp:10942] Removing task 
13880c12-575d-42f2-8bfc-360e9549a694 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 1cd8243a-73d8-454a-8360-9c407e93a3a3- on 
agent 1cd8243a-73d8-454a-8360-9c407e93a3a3-S0 at slave(447)@192.10.1.6:54615 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 17:58:28.611186 21268 master.cpp:1293] Agent 
1cd8243a-73d8-454a-8360-9c407e93a3a3-S0 at slave(447)@192.10.1.6:54615 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 17:58:28.611186 21268 master.cpp:3303] Disconnecting agent 
1cd8243a-73d8-454a-8360-9c407e93a3a3-S0 at slave(447)@192.10.1.6:54615 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 17:58:28.611186 21268 master.cpp:3322] Deactivating agent 
1cd8243a-73d8-454a-8360-9c407e93a3a3-S0 at slave(447)@192.10.1.6:54615 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 17:58

Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/
---

Review request for mesos and Armand Grillet.


Repository: mesos


Description
---

After the user has installed Python 3, if they use the Python 2
scripts, we want to alert them to start using (and therefore testing)
the ported Python 3 scripts.


Diffs
-

  support/check-python3.py PRE-CREATION 


Diff: https://reviews.apache.org/r/67292/diff/1/


Testing
---


Thanks,

Andrew Schwartzmeyer

Re: Review Request 67271: Renamed an ambiguous variable in network port isolator.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67271/
---

(Updated May 24, 2018, 6:17 p.m.)


Review request for mesos and James Peach.


Summary (updated)
-

Renamed an ambiguous variable in network port isolator.


Bugs: MESOS-8340
https://issues.apache.org/jira/browse/MESOS-8340


Repository: mesos


Description (updated)
---

Renamed an ambiguous variable in network port isolator.


Diffs (updated)
-

  src/slave/containerizer/mesos/isolators/network/ports.hpp 
ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
  src/slave/containerizer/mesos/isolators/network/ports.cpp 
1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 


Diff: https://reviews.apache.org/r/67271/diff/3/

Changes: https://reviews.apache.org/r/67271/diff/2-3/


Testing
---


Thanks,

Xudong Ni

Re: Review Request 67274: Rejected non-zero scalar resource values which be represented as zero.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67274/#review203797
---



Patch looks great!

Reviews applied: [67274]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 24, 2018, 2:09 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67274/
> ---
> 
> (Updated May 24, 2018, 2:09 a.m.)
> 
> 
> Review request for mesos, Jie Yu, Joris Van Remoortere, Michael Park, and 
> Neil Conway.
> 
> 
> Bugs: MESOS-8945
> https://issues.apache.org/jira/browse/MESOS-8945
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Internally values of scalar resources are stored in a fixed point
> representation. This can lead to (expected) precision loss.
> 
> This patch adds resource validation ensuring that values of scalar
> resources are either zero or large enough so that they are still
> represented as non-zero values after conversion to the internal fixed
> point format.
> 
> While this patch should not change valid, intended use cases, it might
> change the behavior for erroneous, already broken framework workflows.
> 
> 
> Diffs
> -
> 
>   src/common/resources.cpp 192d086e60b81c0ae8e26d4120beee2034f63d3e 
>   src/tests/master_tests.cpp 6a0fa988aebaf1cf66abaaca0c628f981c6ccc31 
>   src/tests/resources_tests.cpp 1201efec3ff0abc268f896c1fe2330571390b3fd 
> 
> 
> Diff: https://reviews.apache.org/r/67274/diff/2/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67286: White list fds that child processes can inherit in stout.

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67286/#review203795
---

3rdparty/stout/include/stout/internal/windows/inherit.hpp
Lines 23 (patched)

Do you know if this header might include `windows.h`? If it does, we should 
move this `#include` to `stout/windows.hpp` and then include that here to 
preserve Windows header ordering.

3rdparty/stout/include/stout/internal/windows/inherit.hpp
Lines 33 (patched)

const ref?

3rdparty/stout/include/stout/internal/windows/inherit.hpp
Lines 40 (patched)

We should document in a comment the arguments to this system call.

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 252-255 (original), 253-257 (patched)

Just a style suggestion, but we could collapse this to:

```
  DWORD creation_flags = EXTENDED_STARTUPINFO_PRESENT | 
CREATE_UNICODE_ENVIRONMENT;
  if (create_suspended) {
creation_flags |= CREATE_SUSPENDED;
  }
```

and update the comment (or rather, delete the comment because it's quite 
superfluous... my bad, pretty sure I wrote it).

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 274 (patched)

s/inherted/inherited

But really, this comment should describe what we're doing here in detail, 
as it's rather odd.

(1) We're setting the pipe handles and whitelisted handles to be 
temporarily inheritable.
(2) We're explicitly whitelisting the handles using a Windows API.
(3) We're then setting the handles to back to non-inheritable.

Maybe a little more detailed than that too.

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 276-277 (patched)

Style: collapse. I think the line break is from an old iteartion, but the 
type name got wayyy shorter.

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 284 (patched)

I know we're not consistent about this, but let's be explicit here so it's 
clear why we're not just using the `pipes` array:

`handles.emplace_back(static_cast(fd))`.

This will also lessen the work to make the cast operators of `int_fd` 
explicit (which I really should have already done...).

This should also be a line up so that we don't split the `Try` from its 
associated `if (isError())`

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 276-279 (original), 293-296 (patched)

Delete this change.

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 305 (patched)

Same note as above.

3rdparty/stout/include/stout/os/windows/shell.hpp
Line 287 (original), 312 (patched)

Can't we declare and assign here? I don't think we need to declare it above.

3rdparty/stout/include/stout/os/windows/shell.hpp
Lines 320 (patched)

We should set this as soon as we've declared `startup_info_ex`.

- Andrew Schwartzmeyer

On May 24, 2018, 10:50 a.m., Radhika Jandhyala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67286/
> ---
> 
> (Updated May 24, 2018, 10:50 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Li 
> Li, and Radhika Jandhyala.
> 
> 
> Bugs: MESOS-8926
> https://issues.apache.org/jira/browse/MESOS-8926
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> White list fds that child processes can inherit in stout.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/inherit.hpp 
> 7dbde820e775cbaeb8db4bc4559ab432903e75ea 
>   3rdparty/stout/include/stout/os/windows/shell.hpp 
> 8da612af2888ff4d4d458ea5b16cdb08779b6f4c 
> 
> 
> Diff: https://reviews.apache.org/r/67286/diff/1/
> 
> 
> Testing
> ---
> 
> All Mesos-tests
> 
> 
> Thanks,
> 
> Radhika Jandhyala
> 
>

Re: Review Request 67287: White list fds that child processes can inherit in libprocess.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67287/
---

(Updated May 24, 2018, 6:43 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

Libprocess: White list fds that child processes can inherit.


Diffs
-

  3rdparty/libprocess/include/process/subprocess.hpp 
6a1262340c333b617402637e648c12769827ffc4 
  3rdparty/libprocess/src/subprocess.cpp 
d7a725363251f9c54072cd7551f5598696938308 
  3rdparty/libprocess/src/subprocess_windows.hpp 
c7ed0ad18f5b46a1d5ac2a6e51883aefb7c1692f 


Diff: https://reviews.apache.org/r/67287/diff/1/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/
---

(Updated May 24, 2018, 6:43 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

Mesos containerizer.: White list fds that child processes can inherit


Diffs
-

  src/slave/containerizer/mesos/containerizer.cpp 
eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
  src/slave/containerizer/mesos/launcher.hpp 
f69d934d2e1a129e10df8c7f5c78723e832adc7d 
  src/slave/containerizer/mesos/launcher.cpp 
2fe47d368cb82a46328e1f636baa836272db244c 
  src/slave/containerizer/mesos/linux_launcher.hpp 
0ea9b875ae46cadea483bc8dd8bf4907fd324dc9 
  src/slave/containerizer/mesos/linux_launcher.cpp 
80e444501e429c1e1ae354abcd51f86430316ada 


Diff: https://reviews.apache.org/r/67288/diff/1/


Testing
---


Thanks,

Radhika Jandhyala

Re: Review Request 67286: White list fds that child processes can inherit in stout.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67286/
---

(Updated May 24, 2018, 6:44 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

Stout: White list fds that child processes can inherit.


Diffs
-

  3rdparty/stout/include/stout/internal/windows/inherit.hpp 
7dbde820e775cbaeb8db4bc4559ab432903e75ea 
  3rdparty/stout/include/stout/os/windows/shell.hpp 
8da612af2888ff4d4d458ea5b16cdb08779b6f4c 


Diff: https://reviews.apache.org/r/67286/diff/1/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/#review203798
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67286', '67287', '67288']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (105 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (987 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (70 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (711 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (736 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (735 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (761 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (443432 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288/logs/mesos-tests-stderr.log):

```
I0524 18:44:41.671682  9016 master.cpp:10843] Updating the state of task 
3c3864d6-6308-42e5-9a55-48a0a7104eea of framework 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 18:44:41.672682 15844 slave.cpp:3935] Shutting down framework 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-
I0524 18:44:41.672682 158I0524 18:44:41.509665 21876 exec.cpp:162] Version: 
1.7.0
I0524 18:44:41.534665 16468 exec.cpp:236] Executor registered on agent 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-S0
I0524 18:44:41.538671 11924 executor.cpp:178] Received SUBSCRIBED event
I0524 18:44:41.542667 11924 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 18:44:41.543673 11924 executor.cpp:178] Received LAUNCH event
I0524 18:44:41.548682 11924 executor.cpp:665] Starting task 
3c3864d6-6308-42e5-9a55-48a0a7104eea
I0524 18:44:41.626684 11924 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 18:44:41.643685 11924 executor.cpp:678] Forked command at 21096
I0524 18:44:41.675681  8068 exec.cpp:445] Executor asked to shutdown
I0524 18:44:41.675681 19868 executor.cpp:178] Received SHUTDOWN event
I0524 18:44:41.675681 19868 executor.cpp:781] Shutting down
I0524 18:44:41.675681 19868 executor.cpp:894] Sending SIGTERM to process tree 
at pid 2144 slave.cpp:6656] Shutting down executor 
'3c3864d6-6308-42e5-9a55-48a0a7104eea' of framework 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d- at executor(1)@192.10.1.6:57255
I0524 18:44:41.674679  9016 master.cpp:10942] Removing task 
3c3864d6-6308-42e5-9a55-48a0a7104eea with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 5ac27953-d736-4ee8-aa5a-0f94cd927f5d- on 
agent 5ac27953-d736-4ee8-aa5a-0f94cd927f5d-S0 at slave(448)@192.10.1.6:57234 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 18:44:41.674679 15844 slave.cpp:929] Agent terminating
W0524 18:44:41.675681 15844 slave.cpp:3931] Ignoring shutdown framework 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d- because it is terminating
I0524 18:44:41.677678 15756 master.cpp:1293] Agent 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-S0 at slave(448)@192.10.1.6:57234 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 18:44:41.677678 15756 master.cpp:3303] Disconnecting agent 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-S0 at slave(448)@192.10.1.6:57234 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 18:44:41.678678 11896 hierarchical.cpp:344] Removed framework 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-
I0524 18:44:41.678678 15756 master.cpp:3322] Deactivating agent 
5ac27953-d736-4ee8-aa5a-0f94cd927f5d-

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/#review203799
---



Please fix up the commit summary message.


src/slave/containerizer/mesos/containerizer.cpp
Lines 1876-1878 (patched)


This could just be:

`const std::vector whitelist_fds{pipes[0], pipes[1]};



src/slave/containerizer/mesos/launcher.cpp
Lines 95 (patched)


Nit: s/` )`/`)`


- Andrew Schwartzmeyer


On May 24, 2018, 11:43 a.m., Radhika Jandhyala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67288/
> ---
> 
> (Updated May 24, 2018, 11:43 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
> Yu, Li Li, and Radhika Jandhyala.
> 
> 
> Bugs: MESOS-8926
> https://issues.apache.org/jira/browse/MESOS-8926
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Mesos containerizer.: White list fds that child processes can inherit
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
>   src/slave/containerizer/mesos/launcher.hpp 
> f69d934d2e1a129e10df8c7f5c78723e832adc7d 
>   src/slave/containerizer/mesos/launcher.cpp 
> 2fe47d368cb82a46328e1f636baa836272db244c 
>   src/slave/containerizer/mesos/linux_launcher.hpp 
> 0ea9b875ae46cadea483bc8dd8bf4907fd324dc9 
>   src/slave/containerizer/mesos/linux_launcher.cpp 
> 80e444501e429c1e1ae354abcd51f86430316ada 
> 
> 
> Diff: https://reviews.apache.org/r/67288/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Radhika Jandhyala
> 
>

Re: Review Request 67287: White list fds that child processes can inherit in libprocess.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67287/#review203800
---



Please fix up the commit message.

- Andrew Schwartzmeyer


On May 24, 2018, 11:43 a.m., Radhika Jandhyala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67287/
> ---
> 
> (Updated May 24, 2018, 11:43 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
> Yu, Li Li, and Radhika Jandhyala.
> 
> 
> Bugs: MESOS-8926
> https://issues.apache.org/jira/browse/MESOS-8926
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Libprocess: White list fds that child processes can inherit.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/include/process/subprocess.hpp 
> 6a1262340c333b617402637e648c12769827ffc4 
>   3rdparty/libprocess/src/subprocess.cpp 
> d7a725363251f9c54072cd7551f5598696938308 
>   3rdparty/libprocess/src/subprocess_windows.hpp 
> c7ed0ad18f5b46a1d5ac2a6e51883aefb7c1692f 
> 
> 
> Diff: https://reviews.apache.org/r/67287/diff/1/
> 
> 
> Testing
> ---
> 
> All Mesos-tests
> 
> 
> Thanks,
> 
> Radhika Jandhyala
> 
>

Re: Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/
---

(Updated May 24, 2018, 11:58 a.m.)


Review request for mesos and Armand Grillet.


Repository: mesos


Description
---

After the user has installed Python 3, if they use the Python 2
scripts, we want to alert them to start using (and therefore testing)
the ported Python 3 scripts.


Diffs
-

  support/check-python3.py PRE-CREATION 


Diff: https://reviews.apache.org/r/67292/diff/1/


Testing (updated)
---

Non-destructively instructs the user to try the Python 3 scripts.

```
> ./support/apply-reviews.py -r 67285
Congratulations! You have Python 3 installed correctly.
Please start using the scripts in `support/python3`.
Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
so that the Git hooks use the Python 3 scripts.
2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
[8115/8115] -> "67285.patch" [1]
```

```
> ./support/python3/apply-reviews.py -r 67285
2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
[8115/8115] -> "67285.patch" [1]
```

```
> git commit
Usage: ./mesos-split.py ...
Congratulations! You have Python 3 installed correctly.
Please start using the scripts in `support/python3`.
Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
so that the Git hooks use the Python 3 scripts.
On branch python3
```

```
> env MESOSSUPPORTPYTHON=3 git commit
Using Python 3 support scripts to verify what is about to be committed.
No C++ files to lint
No JavaScript files to lint
No Python files to lint
You're using Python 3
```


Thanks,

Andrew Schwartzmeyer

Re: Review Request 67291: Adjusted the tests that use nobody.

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203793
---

It seems to me that most of the modified tests work just fine with `nobody`. 
For example, the `FetcherTest` and `CapabilitiesTest` don't need an 
unpriviledged user that can access test launcher's home. Are we advocating a 
new pattern here and want to avoid `nobody` for all tests?

src/tests/container_logger_tests.cpp
Line 747 (original), 751 (patched)

s/"nobody"/$SUDO_USER/

src/tests/containerizer/docker_containerizer_tests.cpp
Lines 4200 (patched)

Not yours, but how about renaming this test to 
`ROOT_DOCKER_UNPRIVILEGED_USER_NonRootSandbok`?

src/tests/environment.cpp
Lines 871 (patched)

Should we also test that `$SUDO_USER` is not `root`?

src/tests/environment.cpp
Lines 875 (patched)

(From Greg) How about `No usable unpriviledged user found from the 
'SUDO_USER' environment variable.` so one can run the tests as `root` and 
populate it with a non-root user manually if they want to?

src/tests/environment.cpp
Lines 885 (patched)

Not a real issue here, but I'm thinking if just `UNPRIVILEDGED_` is clear 
enough. `UNPRIVILEDGED_USER_` is definitely more elaborative, but it's longer 
and contains a `_` which we also use as keyword deliminators for test filters. 
Although we already have examples like `NVEDIA_GPU_`, I feel that avoiding `_` 
in the matching string would also make it easier to see what filters are 
applyed on a test.

src/tests/slave_tests.cpp
Line 1121 (original), 1072 (patched)

```
../src/tests/slave_tests.cpp:1072:26: warning: adding 'unsigned int' to a 
string does not append to the string [-Wstring-plus-int]
  "test `id -u` == " + uid.get());
  ~~~^~~
```

- Chun-Hung Hsiao

On May 24, 2018, 5:49 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 5:49 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/#review203801
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67247', '67099', '67282', '67059', '67292']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67292

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67292/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (107 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1015 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (71 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (752 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (776 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (735 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (759 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (445090 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67292/logs/mesos-tests-stderr.log):

```
I0524 19:30:29.764704 19840 master.cpp:10843] Updating the state of task 
0bac9c1b-d8b1-4ebb-93b3-6d807febbddc of framework 
e5c864b0-5e59-44c8-8777-2172fc0c5877- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 19:30:29.764704 20852 slave.cpp:3935] Shutting down framework 
e5c864b0-5e59-44c8-8777-2172fc0c5877-
I0524 19:30:29.765713 20852 slave.cpp:6656] Shutting down executor 
'0bac9c1b-d8b1-4ebb-93b3-6d807febbddc' of framework 
e5c864b0-5e59-44c8-8777-2172fc0c5877- at executor(1)@192.10.1.6:59881
I0524 19:30:29.766670 20852 slave.cpp:929] Agent terminating
W0524 19:30:29.767705 20852 slave.cpp:3931] Ignoring shutdown framework 
e5c864b0-5e59-44c8-8777-2172fc0c5877- because it is terminating
I0524 19:30:29.767705 19840 master.cpp:10942] Removing task 
0bac9c1b-d8b1-4ebb-93b3-6d807febbddc with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework e5c864b0-5e59-44c8-8777-2172fc0c5877- on 
agent e5c864b0-5e59-44c8-8777-2172fc0c5877-S0 at slave(448)@192.10.1.6:59I0524 
19:30:29.596686 21692 exec.cpp:162] Version: 1.7.0
I0524 19:30:29.621783 14796 exec.cpp:236] Executor registered on agent 
e5c864b0-5e59-44c8-8777-2172fc0c5877-S0
I0524 19:30:29.625706 22384 executor.cpp:178] Received SUBSCRIBED event
I0524 19:30:29.630690 22384 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 19:30:29.630690 22384 executor.cpp:178] Received LAUNCH event
I0524 19:30:29.635706 22384 executor.cpp:665] Starting task 
0bac9c1b-d8b1-4ebb-93b3-6d807febbddc
I0524 19:30:29.718690 22384 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 19:30:29.735697 22384 executor.cpp:678] Forked command at 10208
I0524 19:30:29.766670 16644 exec.cpp:445] Executor asked to shutdown
I0524 19:30:29.767705 16764 executor.cpp:178] Received SHUTDOWN event
I0524 19:30:29.767705 16764 executor.cpp:781] Shutting down
I0524 19:30:29.767705 16764 executor.cpp:894] Sending SIGTERM to process tree 
at pid 10860 (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 19:30:29.771683 19840 master.cpp:1293] Agent 
e5c864b0-5e59-44c8-8777-2172fc0c5877-S0 at slave(448)@192.10.1.6:59860 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 19:30:29.771683 19840 master.cpp:3303] Disconnecting agent 
e5c864b0-5e59-44c8-8777-2172fc0c5877-S0 at slave(448)@192.10.1.6:59860 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 19:30:29.771683 19916 hierarchical.cpp:344] Removed framework 
e5c864b0-5e59-44c8-8777-2172fc0c5877-
I0524 19:30:29.771683 19840 master.cpp:3322] Deactivating agent 
e5c864b0-5e59-44c8-

Re: Review Request 67291: Adjusted the tests that use nobody.



> On May 24, 2018, 7:29 p.m., Chun-Hung Hsiao wrote:
> > It seems to me that most of the modified tests work just fine with 
> > `nobody`. For example, the `FetcherTest` and `CapabilitiesTest` don't need 
> > an unpriviledged user that can access test launcher's home. Are we 
> > advocating a new pattern here and want to avoid `nobody` for all tests?

using `nobody` is weird because it does not necessarily exist. Using $SUDO_USER 
is better in that case. I think we should avoid using nobody in all our tests.


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203793
---


On May 24, 2018, 5:49 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 5:49 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67291: Adjusted the tests that use nobody.



> On May 24, 2018, 7:29 p.m., Chun-Hung Hsiao wrote:
> > src/tests/environment.cpp
> > Lines 871 (patched)
> > 
> >
> > Should we also test that `$SUDO_USER` is not `root`?

Yes! good catch!


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203793
---


On May 24, 2018, 5:49 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 5:49 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67291: Adjusted the tests that use nobody.



> On May 24, 2018, 7:29 p.m., Chun-Hung Hsiao wrote:
> > src/tests/environment.cpp
> > Lines 885 (patched)
> > 
> >
> > Not a real issue here, but I'm thinking if just `UNPRIVILEDGED_` is 
> > clear enough. `UNPRIVILEDGED_USER_` is definitely more elaborative, but 
> > it's longer and contains a `_` which we also use as keyword deliminators 
> > for test filters. Although we already have examples like `NVEDIA_GPU_`, I 
> > feel that avoiding `_` in the matching string would also make it easier to 
> > see what filters are applyed on a test.

I was using `UNPRIVILEDGED_` initially, but found it quite confusing because 
having `ROOT_UNPRIVILEGED` is definite weird.


- Jie


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203793
---


On May 24, 2018, 5:49 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 5:49 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67264: Unmounted any dangling persistent volume in gc paths.

2018-05-24 Thread Zhitao Li

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67264/
---

(Updated May 24, 2018, 12:48 p.m.)

Review request for mesos, Jason Lai and Jie Yu.

Changes
---

Fix test compilation due to changed constructor interface.

Bugs: MESOS-8830
https://issues.apache.org/jira/browse/MESOS-8830

Repository: mesos

Description
---

In various corner cases, agent may not get chance to properly unmount
persistent volumes mounted inside an executor's sandbox. When GC later
gets to these sandbox directories, permanent data loss can happen (see
MESOS-8830).

This patch added some protection to unmount possible persistent
volumes inside a path to gc, and skipped the path if unmount failed.

NOTE: this means agent will not garbage collect any path if it cannot
read its own `mountinfo` table.

Diffs (updated)
-

src/local/local.cpp afff54653e8e659d947ddbee6dc38ba2715f2a78
src/slave/gc.hpp df40165bb8a23f065156bf6c5f354b143d88c088
src/slave/gc.cpp 390b35e6d17d6614a73c9548decbf10739560106
src/slave/gc_process.hpp 20374ad91820341282fdf18ecade60a020e26cea
src/slave/main.cpp 646125344d590b28256d8ee684d7e51a90e82f23
src/slave/paths.hpp 015896453410a33923eed07b3e676be19af62a48
src/slave/paths.cpp ed0b1276908f4990ce7a24c96aea20e8c79d3126
src/tests/cluster.cpp b56212f6529a4d307e65797ad9bb34f2104fc832
src/tests/gc_tests.cpp 619ed22edd9b3909ea24cdcbf62c354420a8d031
src/tests/mesos.hpp 733344a2f07ebd9d841a55fb9bbfda2e3c1a1eb2
src/tests/mesos.cpp d3c87c295429481c59d5a49398e289a4b84e4496
src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08

Diff: https://reviews.apache.org/r/67264/diff/2/

Changes: https://reviews.apache.org/r/67264/diff/1-2/

Testing
---

Tested with following procedures:
1. Start a test master and agent;
2. Created a persistent volume on agent through operator API;
3. Use `mesos-execute` to run a task;
4. Stop the agent;
5. Manually bind mount persistent volume path into a `volume` directory inside
the executor sandbox (to simulate a dangling mount in MESOS-8830);
6. Restart agent with `--gc_disk_headroom=1.0 --gc_delay=1secs` to force it gc
the path immediately.

With this fix, we observed that the dangling mount is automatically cleaned up,
and agent produces log line:
```
W0523 06:00:04.001075 82745 gc.cpp:229] Unmounting dangling mount point
'/home/zhitao/mesos-workdir/slaves/b3eb3aff-d19d-45ff-8113-f0316462d3fa-S0/frameworks/b3eb3aff-d19d-45ff-8113-f0316462d3fa-/executors/test_id/runs/1cd3bd06-2632-4541-a708-80c7cd51c74b/volume'
of persistent volume '/home/zhitao/mesos-workdir/volumes/roles/role/id1'
inside garbage collected path
'/home/zhitao/mesos-workdir/slaves/b3eb3aff-d19d-45ff-8113-f0316462d3fa-S0'
```

Thanks,

Zhitao Li

Re: Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/
---

(Updated May 24, 2018, 8:19 p.m.)


Review request for mesos, Andrew Schwartzmeyer and James Peach.


Changes
---

Addressed the comments.


Repository: mesos


Description
---

Used `$SUDO_USER` instead because `nobody` sometimes cannot access
direcotries under `$HOME` of the current user running the tests.


Diffs (updated)
-

  src/tests/container_logger_tests.cpp 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
  src/tests/containerizer/capabilities_tests.cpp 
734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
  src/tests/containerizer/cgroups_isolator_tests.cpp 
40c18a1528f2da32c608b6c774500933a08562ea 
  src/tests/containerizer/docker_containerizer_tests.cpp 
e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
  src/tests/containerizer/mesos_containerizer_tests.cpp 
01f2b38cfa67b144298c361e92170322864ac201 
  src/tests/containerizer/provisioner_docker_tests.cpp 
c664ff807583d587d94b0ab797330d5d3daf7657 
  src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
0400052a1c73ccc15c932c573cfb324ca83e2e59 
  src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
  src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
  src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 


Diff: https://reviews.apache.org/r/67291/diff/3/

Changes: https://reviews.apache.org/r/67291/diff/2-3/


Testing
---

sudo make check


Thanks,

Jie Yu

Re: Review Request 67255: Added per-CSI-call RPC metrics for SLRP.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67255/
---

(Updated May 24, 2018, 8:20 p.m.)


Review request for mesos, Benjamin Bannier, Greg Mann, Jie Yu, and Jan Schlicht.


Changes
---

Fixed a typo.


Bugs: MESOS-8943
https://issues.apache.org/jira/browse/MESOS-8943


Repository: mesos


Description
---

For each CSI call, e.g., `csi.v0.Identity.Probe`, we the following
metrics for SLRP:
`csi_plugin/rpcs/csi.v0.Identity.Probe/pending`
`csi_plugin/rpcs/csi.v0.Identity.Probe/successes`
`csi_plugin/rpcs/csi.v0.Identity.Probe/errors`
`csi_plugin/rpcs/csi.v0.Identity.Probe/discarded`

To add these per-CSI-call metrics, we introduced class template
`messo::csi::v0::Client::MethodName` to translate RPC methods of the CSI
client into string names, and added the `call` helper function in SLRP
to intercept CSI calls and update the corresponding metrics.


Diffs (updated)
-

  src/csi/client.hpp 9d7019afb08168d18e5a78831a45af88a75bf809 
  src/csi/client.cpp a4ba1f12a79354f81b29924b19b119fad31a06b9 
  src/resource_provider/storage/provider.cpp 
63b5d7e5f10d6ad02b5cd11b119def3b4abf4180 


Diff: https://reviews.apache.org/r/67255/diff/6/

Changes: https://reviews.apache.org/r/67255/diff/5-6/


Testing
---

sudo make check

A unit test for the metrics will be introduced in the next patch.


Thanks,

Chun-Hung Hsiao

Re: Review Request 67195: Adding enforce_container_ports flag for network ports isolation.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/#review203805
---


Fix it, then Ship it!




Commit subject should be "Added ..."


docs/isolators/network-ports.md
Lines 49 (patched)


How about this:

```
The `--enforce_container_ports` flag, specifies
whether the network ports isolator should terminate
tasks that listen on ports they have not been assigned.
If enforcement is disabled, the isolator will log violations
but will not terminate tasks. By default, network port enforcement
is disabled.
```



src/slave/containerizer/mesos/isolators/network/ports.cpp
Lines 582 (patched)


"while port enforcement is disabled"



src/slave/containerizer/mesos/isolators/network/ports.cpp
Lines 586 (patched)


Newline after the '}'.



src/slave/flags.cpp
Lines 1123 (patched)


"is used by the `network/ports` isolator."

Note the backquotes and fill stop.



src/tests/containerizer/ports_isolator_tests.cpp
Lines 834 (patched)


Needs a full-stop "is false."



src/tests/containerizer/ports_isolator_tests.cpp
Lines 835 (patched)


`ROOT_NC_NoPortEnforcement`



src/tests/containerizer/ports_isolator_tests.cpp
Lines 929 (patched)


Add a comment here:
```
// Since container ports are not being enforced, we expect that the task
// should still be running after the check and that we should be able to
// explicitly kill it.
```


- James Peach


On May 23, 2018, 6:33 p.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67195/
> ---
> 
> (Updated May 23, 2018, 6:33 p.m.)
> 
> 
> Review request for mesos, James Peach and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> To reduce deployment risk, a nonenforce mode is added for network
> port isolator. When this flag is set as false(default is false),
> even task uses ports not in the container resources, the container
> will not raise any limitation.
> 
> Add new test for this flag and update the existing tests
> 
> 
> Diffs
> -
> 
>   docs/isolators/network-ports.md ea63968481ce52c46e0a98e242da49baf6962009 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
>   src/slave/flags.hpp a839591a2b66444ad97fced0620201dde656352d 
>   src/slave/flags.cpp a319b5ea633c41fd8a252c5e1617ac52d1480ba5 
>   src/tests/containerizer/ports_isolator_tests.cpp 
> c5b9f926047792e7f9d1f0937fa5355b1dd77965 
> 
> 
> Diff: https://reviews.apache.org/r/67195/diff/3/
> 
> 
> Testing
> ---
> 
> New test added for the flag; Related unit tests passed.
> 
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_AllocatedPorts (1906 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortsResource (1788 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortEnforement (17001 ms)
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67291: Adjusted the tests that use nobody.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67291/#review203808
---


Fix it, then Ship it!





src/tests/containerizer/provisioner_docker_tests.cpp
Line 957 (original), 957 (patched)


Does it make sense to add double quotes here: `"$SUDO_USER"`?


- Chun-Hung Hsiao


On May 24, 2018, 8:19 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67291/
> ---
> 
> (Updated May 24, 2018, 8:19 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Used `$SUDO_USER` instead because `nobody` sometimes cannot access
> direcotries under `$HOME` of the current user running the tests.
> 
> 
> Diffs
> -
> 
>   src/tests/container_logger_tests.cpp 
> 7ec6f8701ecbc8f43ef41056fcbc17566c7db416 
>   src/tests/containerizer/capabilities_tests.cpp 
> 734aa21c5bda19a1f8d958707249f2dc2b4de6d4 
>   src/tests/containerizer/cgroups_isolator_tests.cpp 
> 40c18a1528f2da32c608b6c774500933a08562ea 
>   src/tests/containerizer/docker_containerizer_tests.cpp 
> e37a9c11fe67846ceeeb6cfbf8a75d1e5753e185 
>   src/tests/containerizer/mesos_containerizer_tests.cpp 
> 01f2b38cfa67b144298c361e92170322864ac201 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> c664ff807583d587d94b0ab797330d5d3daf7657 
>   src/tests/containerizer/volume_sandbox_path_isolator_tests.cpp 
> 0400052a1c73ccc15c932c573cfb324ca83e2e59 
>   src/tests/environment.cpp a5a255fc3e92991a201bb6cb4af9a625805395e0 
>   src/tests/fetcher_tests.cpp 95359ec84ed631bdd7b4fb5192e03f68cc57ed61 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67291/diff/3/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67290: Removed an unneeded test.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67290/#review203809
---


Ship it!




Ship It!

- Chun-Hung Hsiao


On May 24, 2018, 5:49 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67290/
> ---
> 
> (Updated May 24, 2018, 5:49 p.m.)
> 
> 
> Review request for mesos and James Peach.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test is covered by many other tests, and is testing for a
> deprecated feature. Thus remove this test.
> 
> 
> Diffs
> -
> 
>   src/tests/slave_tests.cpp 65d860594572b58a50a89358e31e97fd2a10bf08 
> 
> 
> Diff: https://reviews.apache.org/r/67290/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 67137: Avoided leaking file descriptors in Mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67137/#review203810
---



Patch looks great!

Reviews applied: [67137]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 24, 2018, 11:13 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67137/
> ---
> 
> (Updated May 24, 2018, 11:13 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, Jie Yu, and James Peach.
> 
> 
> Bugs: MESOS-8917
> https://issues.apache.org/jira/browse/MESOS-8917
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Avoided leaking file descriptors in Mesos containerizer.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/launch.cpp 
> f25d90651ef32495c9161c3eaed8a327d1b2b926 
> 
> 
> Diff: https://reviews.apache.org/r/67137/diff/8/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> I still need to confirm this patch in CI on a wider range of scenarios.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67264: Unmounted any dangling persistent volume in gc paths.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67264/#review203811
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67264']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67264

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67264/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (107 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1017 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (71 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (749 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (772 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (729 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (754 ms total)

[--] Global test environment tear-down
[==] 981 tests from 95 test cases ran. (435642 ms total)
[  PASSED  ] 980 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67264/logs/mesos-tests-stderr.log):

```
I0524 20:36:29.580374 18960 master.cpp:10843] Updating the state of task 
df2af7bf-2c19-4766-8bc4-c846bf77e848 of framework 
cf73c05b-2bdf-42f0-9509-75f826e46300- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 20:36:29.580374 23248 slave.cpp:3935] Shutting down framework 
cf73c05b-2bdf-42f0-9509-75f826e46300-
I0524 20:36:29.580374 23248 slave.cpp:6656] Shutting down executor 
'df2af7bf-2c19-4766-8bc4-c846bf77e848' of framework 
cf73c05b-2bdf-42f0-9509-75f826e46300- at executor(1)@192.10.1.6:62490
I0524 20:36:29.582537 23248 slave.cpp:929] Agent terminating
W0524 20:36:29.582537 23248 slave.cpp:3931] Ignoring shutdown framework 
cf73c05b-2bdf-42f0-9509-75f826e46300- because it is terminating
I0524 20:36:29.583359 18960 master.cpp:10942] Removing task 
df2af7bf-2c19-4766-8bc4-c846bf77e848 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000]I0524 20:36:29.415405 19000 exec.cpp:162] Version: 1.7.0
I0524 20:36:29.440357  8332 exec.cpp:236] Executor registered on agent 
cf73c05b-2bdf-42f0-9509-75f826e46300-S0
I0524 20:36:29.443363 13296 executor.cpp:178] Received SUBSCRIBED event
I0524 20:36:29.448391 13296 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 20:36:29.448391 13296 executor.cpp:178] Received LAUNCH event
I0524 20:36:29.453393 13296 executor.cpp:665] Starting task 
df2af7bf-2c19-4766-8bc4-c846bf77e848
I0524 20:36:29.535387 13296 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 20:36:29.553393 13296 executor.cpp:678] Forked command at 7304
I0524 20:36:29.582537 11212 exec.cpp:445] Executor asked to shutdown
I0524 20:36:29.583359 20196 executor.cpp:178] Received SHUTDOWN event
I0524 20:36:29.583359 20196 executor.cpp:781] Shutting down
I0524 20:36:29.583359 20196 executor.cpp:894] Sending SIGTERM to process tree 
at pid 730 of framework cf73c05b-2bdf-42f0-9509-75f826e46300- on agent 
cf73c05b-2bdf-42f0-9509-75f826e46300-S0 at slave(448)@192.10.1.6:62469 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 20:36:29.586357 18960 master.cpp:1293] Agent 
cf73c05b-2bdf-42f0-9509-75f826e46300-S0 at slave(448)@192.10.1.6:62469 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 20:36:29.586357 18960 master.cpp:3303] Disconnecting agent 
cf73c05b-2bdf-42f0-9509-75f826e46300-S0 at slave(448)@192.10.1.6:62469 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 20:36:29.586357 13776 hierarchical.cpp:344] Removed framework 
cf73c05b-2bdf-42f0-9509-75f826e46300-
I0524 20:36:29.587376 18960 master.cpp:3322] Deactivating agent 
cf73c05b-2bdf-42f0-9509-75f826e46300-S0 at slave(448)@1

Re: Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/#review203812
---


Ship it!




Ship It!

- Armand Grillet


On May 24, 2018, 6:58 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67292/
> ---
> 
> (Updated May 24, 2018, 6:58 p.m.)
> 
> 
> Review request for mesos and Armand Grillet.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> After the user has installed Python 3, if they use the Python 2
> scripts, we want to alert them to start using (and therefore testing)
> the ported Python 3 scripts.
> 
> 
> Diffs
> -
> 
>   support/check-python3.py PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67292/diff/1/
> 
> 
> Testing
> ---
> 
> Non-destructively instructs the user to try the Python 3 scripts.
> 
> ```
> > ./support/apply-reviews.py -r 67285
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> 2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
> [8115/8115] -> "67285.patch" [1]
> ```
> 
> ```
> > ./support/python3/apply-reviews.py -r 67285
> 2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
> [8115/8115] -> "67285.patch" [1]
> ```
> 
> ```
> > git commit
> Usage: ./mesos-split.py ...
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> On branch python3
> ```
> 
> ```
> > env MESOSSUPPORTPYTHON=3 git commit
> Using Python 3 support scripts to verify what is about to be committed.
> No C++ files to lint
> No JavaScript files to lint
> No Python files to lint
> You're using Python 3
> ```
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/#review203816
---




support/check-python3.py
Lines 37 (patched)


Please change to be `MESOS_SUPPORT_PYTHON`.



support/check-python3.py
Lines 38 (patched)


Please change to be `MESOS_SUPPORT_PYTHON`.


- Armand Grillet


On May 24, 2018, 6:58 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67292/
> ---
> 
> (Updated May 24, 2018, 6:58 p.m.)
> 
> 
> Review request for mesos and Armand Grillet.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> After the user has installed Python 3, if they use the Python 2
> scripts, we want to alert them to start using (and therefore testing)
> the ported Python 3 scripts.
> 
> 
> Diffs
> -
> 
>   support/check-python3.py PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67292/diff/1/
> 
> 
> Testing
> ---
> 
> Non-destructively instructs the user to try the Python 3 scripts.
> 
> ```
> > ./support/apply-reviews.py -r 67285
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> 2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
> [8115/8115] -> "67285.patch" [1]
> ```
> 
> ```
> > ./support/python3/apply-reviews.py -r 67285
> 2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
> [8115/8115] -> "67285.patch" [1]
> ```
> 
> ```
> > git commit
> Usage: ./mesos-split.py ...
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> On branch python3
> ```
> 
> ```
> > env MESOSSUPPORTPYTHON=3 git commit
> Using Python 3 support scripts to verify what is about to be committed.
> No C++ files to lint
> No JavaScript files to lint
> No Python files to lint
> You're using Python 3
> ```
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Review Request 67298: Fixed a performance issue in collect/await.

2018-05-24 Thread Benjamin Mahler


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67298/
---

Review request for mesos, Chun-Hung Hsiao and Gilbert Song.


Repository: mesos


Description
---

It turns out that even with C++11, some `std::list::size()`
implementations are still linear in complexity.

This was found when running perf against a metrics benchmark
for the master. A follow-up to avoid std::list in the collect.hpp
interfaces entirely is needed.


Diffs
-

  3rdparty/libprocess/include/process/collect.hpp 
7e07b1db8fdb577daa58e83a1295e2d9845ffc31 


Diff: https://reviews.apache.org/r/67298/diff/1/


Testing
---

Before:

```
Registered 2000 frameworks
Finished launching the tasks; Sleep 10 seconds ...
Start collecting metrics ...
v0 '/metrics/snapshot' response took 17.751689014secs
v1 'master::call::GetMetrics' application/x-protobuf response took 
17.523928635secs
v1 'master::call::GetMetrics' application/json response took 18.111901732secs
```

After:

```
Registered 2000 frameworks
Finished launching the tasks; Sleep 10 seconds ...
Start collecting metrics ...
v0 '/metrics/snapshot' response took 1.730948431secs
v1 'master::call::GetMetrics' application/x-protobuf response took 
1.697177667secs
v1 'master::call::GetMetrics' application/json response took 2.160314525secs
```


Thanks,

Benjamin Mahler

Re: Review Request 67298: Fixed a performance issue in collect/await.

2018-05-24 Thread Gilbert Song


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67298/#review203819
---


Ship it!




Ship It!

- Gilbert Song


On May 24, 2018, 2:22 p.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67298/
> ---
> 
> (Updated May 24, 2018, 2:22 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Gilbert Song.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> It turns out that even with C++11, some `std::list::size()`
> implementations are still linear in complexity.
> 
> This was found when running perf against a metrics benchmark
> for the master. A follow-up to avoid std::list in the collect.hpp
> interfaces entirely is needed.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/include/process/collect.hpp 
> 7e07b1db8fdb577daa58e83a1295e2d9845ffc31 
> 
> 
> Diff: https://reviews.apache.org/r/67298/diff/1/
> 
> 
> Testing
> ---
> 
> Before:
> 
> ```
> Registered 2000 frameworks
> Finished launching the tasks; Sleep 10 seconds ...
> Start collecting metrics ...
> v0 '/metrics/snapshot' response took 17.751689014secs
> v1 'master::call::GetMetrics' application/x-protobuf response took 
> 17.523928635secs
> v1 'master::call::GetMetrics' application/json response took 18.111901732secs
> ```
> 
> After:
> 
> ```
> Registered 2000 frameworks
> Finished launching the tasks; Sleep 10 seconds ...
> Start collecting metrics ...
> v0 '/metrics/snapshot' response took 1.730948431secs
> v1 'master::call::GetMetrics' application/x-protobuf response took 
> 1.697177667secs
> v1 'master::call::GetMetrics' application/json response took 2.160314525secs
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67195: Adding enforce_container_ports flag for network ports isolation.



> On May 24, 2018, 5:58 p.m., Gilbert Song wrote:
> > we need to update `configure.md` correspondingly.

Updated configuration/agent.md


- Xudong


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/#review203792
---


On May 23, 2018, 6:33 p.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67195/
> ---
> 
> (Updated May 23, 2018, 6:33 p.m.)
> 
> 
> Review request for mesos, James Peach and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> To reduce deployment risk, a nonenforce mode is added for network
> port isolator. When this flag is set as false(default is false),
> even task uses ports not in the container resources, the container
> will not raise any limitation.
> 
> Add new test for this flag and update the existing tests
> 
> 
> Diffs
> -
> 
>   docs/isolators/network-ports.md ea63968481ce52c46e0a98e242da49baf6962009 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
>   src/slave/flags.hpp a839591a2b66444ad97fced0620201dde656352d 
>   src/slave/flags.cpp a319b5ea633c41fd8a252c5e1617ac52d1480ba5 
>   src/tests/containerizer/ports_isolator_tests.cpp 
> c5b9f926047792e7f9d1f0937fa5355b1dd77965 
> 
> 
> Diff: https://reviews.apache.org/r/67195/diff/3/
> 
> 
> Testing
> ---
> 
> New test added for the flag; Related unit tests passed.
> 
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_AllocatedPorts (1906 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortsResource (1788 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortEnforement (17001 ms)
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67195: Adding enforce_container_ports flag for network ports isolation.



> On May 24, 2018, 8:30 p.m., James Peach wrote:
> > Commit subject should be "Added ..."

subject changed


- Xudong


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/#review203805
---


On May 23, 2018, 6:33 p.m., Xudong Ni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67195/
> ---
> 
> (Updated May 23, 2018, 6:33 p.m.)
> 
> 
> Review request for mesos, James Peach and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-8340
> https://issues.apache.org/jira/browse/MESOS-8340
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> To reduce deployment risk, a nonenforce mode is added for network
> port isolator. When this flag is set as false(default is false),
> even task uses ports not in the container resources, the container
> will not raise any limitation.
> 
> Add new test for this flag and update the existing tests
> 
> 
> Diffs
> -
> 
>   docs/isolators/network-ports.md ea63968481ce52c46e0a98e242da49baf6962009 
>   src/slave/containerizer/mesos/isolators/network/ports.hpp 
> ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
>   src/slave/containerizer/mesos/isolators/network/ports.cpp 
> 1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
>   src/slave/flags.hpp a839591a2b66444ad97fced0620201dde656352d 
>   src/slave/flags.cpp a319b5ea633c41fd8a252c5e1617ac52d1480ba5 
>   src/tests/containerizer/ports_isolator_tests.cpp 
> c5b9f926047792e7f9d1f0937fa5355b1dd77965 
> 
> 
> Diff: https://reviews.apache.org/r/67195/diff/3/
> 
> 
> Testing
> ---
> 
> New test added for the flag; Related unit tests passed.
> 
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_AllocatedPorts (1906 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortsResource (1788 ms)
> [ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortEnforement (17001 ms)
> 
> 
> Thanks,
> 
> Xudong Ni
> 
>

Re: Review Request 67195: Added enforce_container_ports flag for network ports isolation.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/
---

(Updated May 24, 2018, 9:56 p.m.)


Review request for mesos, James Peach and Jiang Yan Xu.


Summary (updated)
-

Added enforce_container_ports flag for network ports isolation.


Bugs: MESOS-8340
https://issues.apache.org/jira/browse/MESOS-8340


Repository: mesos


Description (updated)
---

To reduce deployment risk, a nonenforce mode is added for network
port isolator. When this flag is set as false(default is false),
even task uses ports not in the container resources, the container
will not raise any limitation.

Added new test for this flag and updated the existing tests


Diffs (updated)
-

  docs/configuration/agent.md 962211a54177a54b3e38a93aad9af3c7a0f94ecb 
  docs/isolators/network-ports.md ea63968481ce52c46e0a98e242da49baf6962009 
  src/slave/containerizer/mesos/isolators/network/ports.hpp 
ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
  src/slave/containerizer/mesos/isolators/network/ports.cpp 
1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 
  src/slave/flags.hpp a839591a2b66444ad97fced0620201dde656352d 
  src/slave/flags.cpp a319b5ea633c41fd8a252c5e1617ac52d1480ba5 
  src/tests/containerizer/ports_isolator_tests.cpp 
c5b9f926047792e7f9d1f0937fa5355b1dd77965 


Diff: https://reviews.apache.org/r/67195/diff/4/

Changes: https://reviews.apache.org/r/67195/diff/3-4/


Testing
---

New test added for the flag; Related unit tests passed.

[ OK ] NetworkPortsIsolatorTest.ROOT_NC_AllocatedPorts (1906 ms)
[ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortsResource (1788 ms)
[ OK ] NetworkPortsIsolatorTest.ROOT_NC_NoPortEnforement (17001 ms)


Thanks,

Xudong Ni

Re: Review Request 67271: Renamed an ambiguous variable in network port isolator.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67271/
---

(Updated May 24, 2018, 9:57 p.m.)


Review request for mesos and James Peach.


Bugs: MESOS-8340
https://issues.apache.org/jira/browse/MESOS-8340


Repository: mesos


Description
---

Renamed an ambiguous variable in network port isolator.


Diffs (updated)
-

  src/slave/containerizer/mesos/isolators/network/ports.hpp 
ba71087194a3ae74c7e40dffa9c108b02ffa10ad 
  src/slave/containerizer/mesos/isolators/network/ports.cpp 
1f84ed4fb2a30fd095e2faec1038de1fa19a15c5 


Diff: https://reviews.apache.org/r/67271/diff/5/

Changes: https://reviews.apache.org/r/67271/diff/4-5/


Testing
---


Thanks,

Xudong Ni

Re: Review Request 67298: Fixed a performance issue in collect/await.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67298/#review203823
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67298

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67298/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (106 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1015 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (70 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (736 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (763 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (729 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (753 ms total)

[--] Global test environment tear-down
[==] 980 tests from 95 test cases ran. (444938 ms total)
[  PASSED  ] 979 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67298/logs/mesos-tests-stderr.log):

```
I0524 22:15:41.169273 22800 master.cpp:10843] Updating the state of task 
a7dd3100-fa3d-4680-a658-6e901a9f8e88 of framework 
0ec974ec-eae4-4aea-95b1-f539abc239af- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 22:15:41.169273 16100 slave.cpp:3935] Shutting down framework 
0ec974ec-eae4-4aea-95b1-f539abc239af-
I0524 22:15:41.169273 16100 slave.cpp:6656] Shutting down executor 
'a7dd3100-fa3d-4680-a658-6e901a9f8e88' of framework 
0ec974ec-eae4-4aea-95b1-f539abc239af- at executor(1)@192.10.1.6:65103
I0524 22:15:41.171253 16100 slave.cpp:929] Agent terminating
W0524 22:15:41.171253 16100 slave.cpp:3931] Ignoring shutdown framework 
0ec974ec-eae4-4aea-95b1-f539abc239af- because it is terminating
I0524 22:15:41.171253 22800 master.cpp:10942] Removing task 
a7dd3100-fa3d-4680-a658-6e901a9f8e88 wiI0524 22:15:41.003270 22036 
exec.cpp:162] Version: 1.7.0
I0524 22:15:41.028254 20288 exec.cpp:236] Executor registered on agent 
0ec974ec-eae4-4aea-95b1-f539abc239af-S0
I0524 22:15:41.032265 20576 executor.cpp:178] Received SUBSCRIBED event
I0524 22:15:41.038264 20576 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 22:15:41.039264 20576 executor.cpp:178] Received LAUNCH event
I0524 22:15:41.043264 20576 executor.cpp:665] Starting task 
a7dd3100-fa3d-4680-a658-6e901a9f8e88
I0524 22:15:41.123265 20576 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 22:15:41.141259 20576 executor.cpp:678] Forked command at 5872
I0524 22:15:41.171253 21828 exec.cpp:445] Executor asked to shutdown
I0524 22:15:41.171253 17672 executor.cpp:178] Received SHUTDOWN event
I0524 22:15:41.171253 17672 executor.cpp:781] Shutting down
I0524 22:15:41.171253 17672 executor.cpp:894] Sending SIGTERM to process tree 
at pid 587th resources cpus(allocated: *):4; mem(allocated: *):2048; 
disk(allocated: *):1024; ports(allocated: *):[31000-32000] of framework 
0ec974ec-eae4-4aea-95b1-f539abc239af- on agent 
0ec974ec-eae4-4aea-95b1-f539abc239af-S0 at slave(447)@192.10.1.6:65082 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 22:15:41.174266 22800 master.cpp:1293] Agent 
0ec974ec-eae4-4aea-95b1-f539abc239af-S0 at slave(447)@192.10.1.6:65082 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 22:15:41.175266 22800 master.cpp:3303] Disconnecting agent 
0ec974ec-eae4-4aea-95b1-f539abc239af-S0 at slave(447)@192.10.1.6:65082 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 22:15:41.175266 22800 master.cpp:3322] Deactivating agent 
0ec974ec-eae4-4aea-95b1-f539abc239af-S0 at slave(447)@192.10.1.6:65082 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 22:15:41.175266 22712 hierarchical.cpp:344] Removed

Re: Review Request 67292: Added warning to Python version checker script.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67292/
---

(Updated May 24, 2018, 3:20 p.m.)


Review request for mesos and Armand Grillet.


Changes
---

Added underscores.


Repository: mesos


Description
---

After the user has installed Python 3, if they use the Python 2
scripts, we want to alert them to start using (and therefore testing)
the ported Python 3 scripts.


Diffs (updated)
-

  support/check-python3.py PRE-CREATION 


Diff: https://reviews.apache.org/r/67292/diff/2/

Changes: https://reviews.apache.org/r/67292/diff/1-2/


Testing
---

Non-destructively instructs the user to try the Python 3 scripts.

```
> ./support/apply-reviews.py -r 67285
Congratulations! You have Python 3 installed correctly.
Please start using the scripts in `support/python3`.
Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
so that the Git hooks use the Python 3 scripts.
2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
[8115/8115] -> "67285.patch" [1]
```

```
> ./support/python3/apply-reviews.py -r 67285
2018-05-24 11:00:25 URL:https://reviews.apache.org/r/67285/diff/raw/ 
[8115/8115] -> "67285.patch" [1]
```

```
> git commit
Usage: ./mesos-split.py ...
Congratulations! You have Python 3 installed correctly.
Please start using the scripts in `support/python3`.
Please also set the environment variable `MESOSSUPPORTPYTHON` to `3`
so that the Git hooks use the Python 3 scripts.
On branch python3
```

```
> env MESOSSUPPORTPYTHON=3 git commit
Using Python 3 support scripts to verify what is about to be committed.
No C++ files to lint
No JavaScript files to lint
No Python files to lint
You're using Python 3
```


Thanks,

Andrew Schwartzmeyer

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/#review203824
---




src/slave/containerizer/mesos/containerizer.cpp
Lines 1878 (patched)


Oh, also, we only need to send `pipes[0]`; as the child process only needs 
the `read` pipe. If I understand this correctly, this control pipe is for the 
parent to signal the child; that is, that child waits in a `read` on the read 
end of the pipe, for the parent to `write` to the write end.

So `std::vector whitelist_fds{pipes[0]};`


- Andrew Schwartzmeyer


On May 24, 2018, 11:43 a.m., Radhika Jandhyala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67288/
> ---
> 
> (Updated May 24, 2018, 11:43 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
> Yu, Li Li, and Radhika Jandhyala.
> 
> 
> Bugs: MESOS-8926
> https://issues.apache.org/jira/browse/MESOS-8926
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Mesos containerizer.: White list fds that child processes can inherit
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
>   src/slave/containerizer/mesos/launcher.hpp 
> f69d934d2e1a129e10df8c7f5c78723e832adc7d 
>   src/slave/containerizer/mesos/launcher.cpp 
> 2fe47d368cb82a46328e1f636baa836272db244c 
>   src/slave/containerizer/mesos/linux_launcher.hpp 
> 0ea9b875ae46cadea483bc8dd8bf4907fd324dc9 
>   src/slave/containerizer/mesos/linux_launcher.cpp 
> 80e444501e429c1e1ae354abcd51f86430316ada 
> 
> 
> Diff: https://reviews.apache.org/r/67288/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Radhika Jandhyala
> 
>

Re: Review Request 63280: Made the containerizer launch be explicit about O_CLOEXEC.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63280/#review203825
---




src/slave/containerizer/mesos/containerizer.cpp
Lines 1815-1817 (original), 1818-1823 (patched)


Just a heads up, this chain is making related changes: 
https://reviews.apache.org/r/67288/



src/slave/containerizer/mesos/containerizer.cpp
Lines 1820 (patched)


Shouldn't we also close the parent's copy of the read end of the pipe 
immediately (and unconditionally) after forking? (Similar to what we do for the 
in/out/err pipes.)


- Andrew Schwartzmeyer


On May 24, 2018, 2:16 p.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63280/
> ---
> 
> (Updated May 24, 2018, 2:16 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Benjamin 
> Mahler, Greg Mann, and Jie Yu.
> 
> 
> Bugs: MESOS-8128
> https://issues.apache.org/jira/browse/MESOS-8128
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Since the containerizer launch depends on the inherited pipe to signal
> the forked child, be explicit about setting O_CLOEXEC on the pipe file
> descriptors. Make sure to close the pipe on the error paths.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 100e3bbda543d87808da9ff6bea42da5099ea8c5 
> 
> 
> Diff: https://reviews.apache.org/r/63280/diff/2/
> 
> 
> Testing
> ---
> 
> make check (Fedora 25)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 67287: White list fds that child processes can inherit in libprocess.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67287/
---

(Updated May 24, 2018, 10:47 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

White list fds that child processes can inherit in libprocess.


Diffs (updated)
-

  3rdparty/libprocess/include/process/subprocess.hpp 
6a1262340c333b617402637e648c12769827ffc4 
  3rdparty/libprocess/src/subprocess.cpp 
d7a725363251f9c54072cd7551f5598696938308 
  3rdparty/libprocess/src/subprocess_windows.hpp 
c7ed0ad18f5b46a1d5ac2a6e51883aefb7c1692f 


Diff: https://reviews.apache.org/r/67287/diff/2/

Changes: https://reviews.apache.org/r/67287/diff/1-2/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/
---

(Updated May 24, 2018, 10:47 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

White list fds that child processes can inherit in mesos containerizer.


Diffs (updated)
-

  src/slave/containerizer/mesos/containerizer.cpp 
eac1d16f2388385fec04ff8f013ce0ebf4e97f0f 
  src/slave/containerizer/mesos/launcher.hpp 
f69d934d2e1a129e10df8c7f5c78723e832adc7d 
  src/slave/containerizer/mesos/launcher.cpp 
2fe47d368cb82a46328e1f636baa836272db244c 
  src/slave/containerizer/mesos/linux_launcher.hpp 
0ea9b875ae46cadea483bc8dd8bf4907fd324dc9 
  src/slave/containerizer/mesos/linux_launcher.cpp 
80e444501e429c1e1ae354abcd51f86430316ada 


Diff: https://reviews.apache.org/r/67288/diff/2/

Changes: https://reviews.apache.org/r/67288/diff/1-2/


Testing
---


Thanks,

Radhika Jandhyala

Re: Review Request 67286: White list fds that child processes can inherit in stout.

2018-05-24 Thread Radhika Jandhyala via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67286/
---

(Updated May 24, 2018, 10:47 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
Yu, Li Li, and Radhika Jandhyala.


Bugs: MESOS-8926
https://issues.apache.org/jira/browse/MESOS-8926


Repository: mesos


Description (updated)
---

White list fds that child processes can inherit in stout.


Diffs (updated)
-

  3rdparty/stout/include/stout/internal/windows/inherit.hpp 
7dbde820e775cbaeb8db4bc4559ab432903e75ea 
  3rdparty/stout/include/stout/os/windows/shell.hpp 
8da612af2888ff4d4d458ea5b16cdb08779b6f4c 


Diff: https://reviews.apache.org/r/67286/diff/2/

Changes: https://reviews.apache.org/r/67286/diff/1-2/


Testing
---

All Mesos-tests


Thanks,

Radhika Jandhyala

Re: Review Request 67286: White list fds that child processes can inherit in stout.

2018-05-24 Thread Radhika Jandhyala via Review Board



> On May 24, 2018, 6:42 p.m., Andrew Schwartzmeyer wrote:
> > 3rdparty/stout/include/stout/internal/windows/inherit.hpp
> > Lines 23 (patched)
> > 
> >
> > Do you know if this header might include `windows.h`? If it does, we 
> > should move this `#include` to `stout/windows.hpp` and then include that 
> > here to preserve Windows header ordering.

It does not include windows.h


- Radhika


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67286/#review203795
---


On May 24, 2018, 10:47 p.m., Radhika Jandhyala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67286/
> ---
> 
> (Updated May 24, 2018, 10:47 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Eric Mumau, Jie 
> Yu, Li Li, and Radhika Jandhyala.
> 
> 
> Bugs: MESOS-8926
> https://issues.apache.org/jira/browse/MESOS-8926
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> White list fds that child processes can inherit in stout.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/inherit.hpp 
> 7dbde820e775cbaeb8db4bc4559ab432903e75ea 
>   3rdparty/stout/include/stout/os/windows/shell.hpp 
> 8da612af2888ff4d4d458ea5b16cdb08779b6f4c 
> 
> 
> Diff: https://reviews.apache.org/r/67286/diff/2/
> 
> 
> Testing
> ---
> 
> All Mesos-tests
> 
> 
> Thanks,
> 
> Radhika Jandhyala
> 
>

Review Request 67301: Fixed a typo in libprocess documentation.

2018-05-24 Thread Meng Zhu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67301/
---

Review request for mesos and Benjamin Mahler.


Repository: mesos


Description
---

Fixed a typo in libprocess documentation.


Diffs
-

  3rdparty/libprocess/README.md d8b28fae4050a71b1a62fda3e13f2f27e176b121 


Diff: https://reviews.apache.org/r/67301/diff/1/


Testing
---

Not needed.


Thanks,

Meng Zhu

Re: Review Request 67280: Added libprocess example code to build.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67280/#review203827
---



Patch looks great!

Reviews applied: [67281, 67270, 67280]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On May 24, 2018, 11:52 a.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67280/
> ---
> 
> (Updated May 24, 2018, 11:52 a.m.)
> 
> 
> Review request for mesos and Benno Evers.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added libprocess example code to build.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/CMakeLists.txt 38522b1091a99141f75eccd7b54152bb40d00939 
>   3rdparty/libprocess/Makefile.am b1d77f3a7ab1426d873d1f1d886e3c255ce32379 
>   3rdparty/libprocess/examples/CMakeLists.txt PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67280/diff/1/
> 
> 
> Testing
> ---
> 
> `make check` (autotools, cmake)
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 67195: Added enforce_container_ports flag for network ports isolation.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67195/#review203828
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67271', '67195']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67195

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67195/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (105 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1002 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (32 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (38 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (776 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (800 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (736 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (760 ms total)

[--] Global test environment tear-down
[==] 980 tests from 95 test cases ran. (442527 ms total)
[  PASSED  ] 979 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67195/logs/mesos-tests-stderr.log):

```
I0524 23:01:16.902797 22300 master.cpp:10843] Updating the state of task 
e71bb8dd-e521-401a-a53b-baca9459eee3 of framework 
cb9a7a95-8ddb-4828-9b88-c1e928650749- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0524 23:01:16.902797 21544 slave.cpp:3935] Shutting down framework 
cb9a7a95-8ddb-4828-9b88-c1e928650749-
I0524 23:01:16.903744 21544 slave.cpp:6656] ShI0524 23:01:16.740767 19640 
exec.cpp:162] Version: 1.7.0
I0524 23:01:16.765769 19524 exec.cpp:236] Executor registered on agent 
cb9a7a95-8ddb-4828-9b88-c1e928650749-S0
I0524 23:01:16.769767 11824 executor.cpp:178] Received SUBSCRIBED event
I0524 23:01:16.773767 11824 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0524 23:01:16.774758 11824 executor.cpp:178] Received LAUNCH event
I0524 23:01:16.778764 11824 executor.cpp:665] Starting task 
e71bb8dd-e521-401a-a53b-baca9459eee3
I0524 23:01:16.860759 11824 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0524 23:01:16.877769 11824 executor.cpp:678] Forked command at 20588
I0524 23:01:16.905735 20256 exec.cpp:445] Executor asked to shutdown
I0524 23:01:16.905735  7304 executor.cpp:178] Received SHUTDOWN event
I0524 23:01:16.906738  7304 executor.cpp:781] Shutting down
I0524 23:01:16.906738  7304 executor.cpp:894] Sending SIGTERM to process tree 
at pid 20utting down executor 'e71bb8dd-e521-401a-a53b-baca9459eee3' of 
framework cb9a7a95-8ddb-4828-9b88-c1e928650749- at 
executor(1)@192.10.1.6:51348
I0524 23:01:16.905735 21544 slave.cpp:929] Agent terminating
I0524 23:01:16.905735 22300 master.cpp:10942] Removing task 
e71bb8dd-e521-401a-a53b-baca9459eee3 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework cb9a7a95-8ddb-4828-9b88-c1e928650749- on 
agent cb9a7a95-8ddb-4828-9b88-c1e928650749-S0 at slave(447)@192.10.1.6:51327 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
W0524 23:01:16.905735 21544 slave.cpp:3931] Ignoring shutdown framework 
cb9a7a95-8ddb-4828-9b88-c1e928650749- because it is terminating
I0524 23:01:16.908730 21328 master.cpp:1293] Agent 
cb9a7a95-8ddb-4828-9b88-c1e928650749-S0 at slave(447)@192.10.1.6:51327 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0524 23:01:16.908730 21328 master.cpp:3303] Disconnecting agent 
cb9a7a95-8ddb-4828-9b88-c1e928650749-S0 at slave(447)@192.10.1.6:51327 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0524 23:01:16.908730 15436 hierarchical.cpp:344] Removed framework 
cb9a7a95-8ddb-4828-9b88-c1e928650749-
I0524 23:01:16.909746 21328 master.cpp:3322] Deactivating agent 
cb9a7a95-8ddb-4828-9b88-c1e928650749-S0 at sl

Re: Review Request 67301: Fixed a typo in libprocess documentation.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67301/#review203830
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67301']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67301

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67301/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (110 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1018 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (35 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (36 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (732 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (758 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (724 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (750 ms total)

[--] Global test environment tear-down
[==] 980 tests from 95 test cases ran. (465868 ms total)
[  PASSED  ] 979 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67301/logs/mesos-tests-stderr.log):

```
I0525 00:07:00.182763 19056 hierarchical.cpp:405] Deactivated framework 
c7d89a87-c885-418c-a13d-aa0290e789e7-
I0525 00:07:00.182763 20412 slave.cpp:3935] Shutting down framework 
c7d89a87-c885-418c-a13d-aa0290e789e7-
I0525 00:07:00.182763 20412 slave.cpp:6656] Shutting down executor 
'f58ace8f-75c7-4049-bccd-87c4e5bbdd54' of framework 
c7d89a87-c885-418c-a13d-aa0290e789e7- at executor(1)@192.10.1.5:53699
I0525 00:07:00.184765 20412 slave.cpp:929] Agent terminating
W0525 00:07:00.184765 20412 slave.cpp:3931] Ignoring shutdown framework 
c7d89a87-c885-418c-a13d-aa0290e789eI0525 00:07:00.016777 22072 exec.cpp:162] 
Version: 1.7.0
I0525 00:07:00.042774 21684 exec.cpp:236] Executor registered on agent 
c7d89a87-c885-418c-a13d-aa0290e789e7-S0
I0525 00:07:00.046763 15720 executor.cpp:178] Received SUBSCRIBED event
I0525 00:07:00.051787 15720 executor.cpp:182] Subscribed executor on 
windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0525 00:07:00.051787 15720 executor.cpp:178] Received LAUNCH event
I0525 00:07:00.056757 15720 executor.cpp:665] Starting task 
f58ace8f-75c7-4049-bccd-87c4e5bbdd54
I0525 00:07:00.135763 15720 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0525 00:07:00.152765 15720 executor.cpp:678] Forked command at 24540
I0525 00:07:00.184765 24176 exec.cpp:445] Executor asked to shutdown
I0525 00:07:00.185768 18720 executor.cpp:178] Received SHUTDOWN event
I0525 00:07:00.185768 18720 executor.cpp:781] Shutting down
I0525 00:07:00.185768 18720 executor.cpp:894] Sending SIGTERM to process tree 
at pid 247- because it is terminating
I0525 00:07:00.185768 23896 master.cpp:10942] Removing task 
f58ace8f-75c7-4049-bccd-87c4e5bbdd54 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework c7d89a87-c885-418c-a13d-aa0290e789e7- on 
agent c7d89a87-c885-418c-a13d-aa0290e789e7-S0 at slave(447)@192.10.1.5:53678 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0525 00:07:00.188784 19056 master.cpp:1293] Agent 
c7d89a87-c885-418c-a13d-aa0290e789e7-S0 at slave(447)@192.10.1.5:53678 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0525 00:07:00.188784 19056 master.cpp:3303] Disconnecting agent 
c7d89a87-c885-418c-a13d-aa0290e789e7-S0 at slave(447)@192.10.1.5:53678 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0525 00:07:00.188784 19056 master.cpp:3322] Deactivating agent 
c7d89a87-c885-418c-a13d-aa0290e789e7-S0 at slave(447)@192.10.1.5:53678 
(windows-01.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0525 00:07:00.188784 16956 hierarchical.cpp:344] Removed framework 
c7d89a87-c885-418c-a13d-aa0290e789e7-
I0525 00:07:00.189766 16956 hier

Re: Review Request 67256: Added a unit test for CSI plugin RPC metrics.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67256/
---

(Updated May 25, 2018, 12:24 a.m.)


Review request for mesos, Benjamin Bannier, Greg Mann, Jie Yu, and Jan Schlicht.


Changes
---

Renamed the `discarded` metric to `cancelled`.


Bugs: MESOS-8943
https://issues.apache.org/jira/browse/MESOS-8943


Repository: mesos


Description
---

This patch adds the `ROOT_CsiPluginRpcMetrics` test that issues a
`CREATE_VOLUME` followed by a `DESTROY_VOLUME`, which would fail due to
an out-of-band deletion of the actual volume.


Diffs (updated)
-

  src/tests/storage_local_resource_provider_tests.cpp 
45cb38922f78941e82667a60b3b71ce220c9202f 


Diff: https://reviews.apache.org/r/67256/diff/2/

Changes: https://reviews.apache.org/r/67256/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 63280: Made the containerizer launch be explicit about O_CLOEXEC.



> On May 24, 2018, 10:36 p.m., Andrew Schwartzmeyer wrote:
> > src/slave/containerizer/mesos/containerizer.cpp
> > Lines 1820 (patched)
> > 
> >
> > Shouldn't we also close the parent's copy of the read end of the pipe 
> > immediately (and unconditionally) after forking? (Similar to what we do for 
> > the in/out/err pipes.)

The parent still needs this pipe. However, as @jieyu points out offline, this 
is still racey, so the right approach is to find some way to apply the 
`UNSET_CLOEXEC` child hook.


- James


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63280/#review203825
---


On May 24, 2018, 9:16 p.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63280/
> ---
> 
> (Updated May 24, 2018, 9:16 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Benjamin 
> Mahler, Greg Mann, and Jie Yu.
> 
> 
> Bugs: MESOS-8128
> https://issues.apache.org/jira/browse/MESOS-8128
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Since the containerizer launch depends on the inherited pipe to signal
> the forked child, be explicit about setting O_CLOEXEC on the pipe file
> descriptors. Make sure to close the pipe on the error paths.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> 100e3bbda543d87808da9ff6bea42da5099ea8c5 
> 
> 
> Diff: https://reviews.apache.org/r/63280/diff/2/
> 
> 
> Testing
> ---
> 
> make check (Fedora 25)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 67288: White list fds that child processes can inherit in mesos containerizer.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67288/#review203832
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67286', '67287', '67288']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288/logs/mesos-tests-stdout.log):

```
[   OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (108 ms)
[--] 9 tests from Endpoint/SlaveEndpointTest (1025 ms total)

[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (34 
ms)
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (36 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (72 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (727 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (754 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (729 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (754 ms total)

[--] Global test environment tear-down
[==] 980 tests from 95 test cases ran. (442674 ms total)
[  PASSED  ] 979 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] 
DockerContainerizerHealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange

 1 FAILED TEST
  YOU HAVE 220 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67288/logs/mesos-tests-stderr.log):

```
I0525 00:30:38.493302 21160 executor.cpp:178] Received SUBSCRIBED event
I0525 00:30:38.498245 21160 executor.cpp:182] Subscribed executor on 
windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0525 00:30:38.498245 21160 executor.cpp:178] Received LAUNCH event
I0525 00:30:38.503278 21160 executor.cpp:665] Starting task 
de5a1f8a-726a-4056-ad9e-288af145ba59
I0525 00:30:38.584270 21160 executor.cpp:485] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I0525 00:30:38.601246 21160 executor.cpp:678] Forked command at 15696
I0525 00:30:38.630240 23368 exec.cpp:445] Executor asked to shutdown
I0525 00:30:38.631242 18588 executor.cpp:178] Received SHUTDOWN event
I0525 00:30:38.631242 18588 executor.cpp:781] Shutting down
I0525 00:30:38.631242 18588 executor.cpp:894] Sending SIGTERM to process tree 
at pid 1581  8372 master.cpp:3266] Deactivating framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea- (default) at 
scheduler-5ee620a2-bdf3-4cb6-b7b6-33dceb06fc4a@192.10.1.6:56147
I0525 00:30:38.629253 16416 hierarchical.cpp:405] Deactivated framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea-
I0525 00:30:38.629253  8372 master.cpp:10843] Updating the state of task 
de5a1f8a-726a-4056-ad9e-288af145ba59 of framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I0525 00:30:38.629253 16788 slave.cpp:3935] Shutting down framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea-
I0525 00:30:38.629253 16788 slave.cpp:6656] Shutting down executor 
'de5a1f8a-726a-4056-ad9e-288af145ba59' of framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea- at executor(1)@192.10.1.6:56168
I0525 00:30:38.630240 16788 slave.cpp:929] Agent terminating
W0525 00:30:38.631242 16788 slave.cpp:3931] Ignoring shutdown framework 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea- because it is terminating
I0525 00:30:38.631242  8372 master.cpp:10942] Removing task 
de5a1f8a-726a-4056-ad9e-288af145ba59 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 06ac683f-0b03-4f54-b4f7-879e8e0fb5ea- on 
agent 06ac683f-0b03-4f54-b4f7-879e8e0fb5ea-S0 at slave(447)@192.10.1.6:56147 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0525 00:30:38.635247  8372 master.cpp:1293] Agent 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea-S0 at slave(447)@192.10.1.6:56147 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net) disconnected
I0525 00:30:38.635247  8372 master.cpp:3303] Disconnecting agent 
06ac683f-0b03-4f54-b4f7-879e8e0fb5ea-S0 at slave(447)@192.10.1.6:56147 
(windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0525 00:30:38.635247  8372 master.cpp:3322] Deactivating agent 
06ac683f-0b03-4f54-b4f7-879e8e0f

Re: Review Request 67303: Added documentation for resource provider and CSI plugin metrics.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67303/
---

(Updated May 25, 2018, 1:05 a.m.)


Review request for mesos, Benjamin Bannier, Greg Mann, and Jie Yu.


Changes
---

Fixed a link to `resource-provider.md` (which is still not existed yet).


Repository: mesos


Description
---

Added documentation for resource provider and CSI plugin metrics.


Diffs (updated)
-

  docs/home.md 5471c70d4023f88adae2234aec267e4d6fea83df 
  docs/monitoring.md c20b99eb32a50830ea6f66b5bdbfcabe45fc6ecb 


Diff: https://reviews.apache.org/r/67303/diff/2/

Changes: https://reviews.apache.org/r/67303/diff/1-2/


Testing
---

Spell checked through VIM's `:set spell` and manually tested the links.


Thanks,

Chun-Hung Hsiao

Review Request 67303: Added documentation for resource provider and CSI plugin metrics.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67303/
---

Review request for mesos, Benjamin Bannier, Greg Mann, and Jie Yu.


Repository: mesos


Description
---

Added documentation for resource provider and CSI plugin metrics.


Diffs
-

  docs/home.md 5471c70d4023f88adae2234aec267e4d6fea83df 
  docs/monitoring.md c20b99eb32a50830ea6f66b5bdbfcabe45fc6ecb 


Diff: https://reviews.apache.org/r/67303/diff/1/


Testing
---

Spell checked through VIM's `:set spell` and manually tested the links.


Thanks,

Chun-Hung Hsiao

Re: Review Request 65112: Added documentation about standalone containers.