Re: Review Request 68548: Added allocator benchmark test harness.

2018-08-28 Thread Meng Zhu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68548/#review208063
---



Glad to see this being added! Thank you!

I did a pass on the test harness. Will look into the test after the harness is 
updated.


src/tests/hierarchical_allocator_benchmarks.cpp
Lines 58 (patched)


Do we need this? Can you audit other "using/include" and remove any that 
are not needed?



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 82 (patched)


`set` would enable us to benchmark multi-role frameworks.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 85 (patched)


Take reference?



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 91 (patched)


We can use `CHECK_NOTERROR` instead of the unguarded `get`.

Moreover, I think it is more flexible to just take a `Resource` object 
here, leaving the parsing or whatever to the caller. For non-trivial resource 
objects e.g. involving persistent volume, they are more likely to already be 
resource objects instead of a plain string.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 109 (patched)


ditto resources object.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 135-136 (patched)


Comparing to the original test harness, we probably want to control more 
things e.g. random allocator and other flag values in the future.

I suggest selectively keep/expose several fields such as interval, sorter 
type and etc. explicitly instead of keeping a default master flag. See my 
comments regarding `initializeCluster` below.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 155 (patched)


Are we relying on callers to name their frameworks ...%pid? Then this is 
very fragile. One solution is to just concatenate with `_#` and document it 
well. I think "nextFrameworkId" is used for that?



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 160 (patched)


ditto



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 162 (patched)


If we `push_back` last, we can std::move.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 167 (patched)


I would expect `frameworkTasksLaunched` tracks 
the number of "launched tasks", not "tasks to launch".

Can you also add some comments to the fields whose meaning is not 
immediately obvious?



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 183 (patched)


ditto



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 190 (patched)


We can std::move here, even in the test, savings are savings:)



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 197 (patched)


I think this TODO is no longer relavent.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 199-200 (patched)


As mentioned above, let's expose more knobs here. For now, maybe just 
allocator type, allocation interval, offercallback and any other that are 
currently being touched.

This will likely make the initialize function signature ugly. It is 
probably better to introduce a flag (maybe utilize the `BenchmarkConfig` 
below?). You probably can come up with a better solution.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 220 (patched)


ah, one thing I forget to mention in the original patch. This vector here 
is fragile. It will be concurrently accessed by both allocator actor and the 
test actor. Like the test base, let's use an asynchronous queue here.


https://github.com/apache/mesos/blob/master/src/tests/hierarchical_allocator_tests.cpp#L282



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 241 (patched)


Create frameworks and agents. (Or just remove the comment).



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 345-346 (patched)


The meaning of these fields are not obvious, adding some comments would be 
helpful.



src/tests/hierarchical_allocator_benchmarks.cpp
Lines 350 (patched)


hmm, just to understand the test 

Re: Review Request 68553: Set 'Connection: close' in the master's streaming API responses.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68553/#review208076
---



PASS: Mesos patch 68553 was successfully built and tested.

Reviews applied: `['68553']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2256/mesos-review-68553

- Mesos Reviewbot Windows


On Aug. 29, 2018, 2:03 a.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68553/
> ---
> 
> (Updated Aug. 29, 2018, 2:03 a.m.)
> 
> 
> Review request for mesos, Gastón Kleiman and Vinod Kone.
> 
> 
> Bugs: MESOS-9189
> https://issues.apache.org/jira/browse/MESOS-9189
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We've seen some HTTP intermediaries (e.g. ELB) decide to re-use
> connections to mesos as an optimization to avoid re-connection
> overhead. As a result, when the end-client of the streaming API
> disconnects from the intermediary, the intermediary leaves the
> connection to Mesos open in an attempt to re-use the connection
> for another request once the response completes. Mesos then thinks
> that the subscriber never disconnected and the intermediary happily
> continues to read the streaming events even though there's no
> end-client.
> 
> To help indicate to intermediaries that the connection SHOULD NOT
> be re-used, we can set the 'Connection: close' header for streaming
> API responses. It may not be respected (since the language seems to
> be SHOULD NOT), but some intermediaries may respect it and close the
> connection if the end-client disconnects.
> 
> Note that libprocess' http server currently doesn't close the the
> connection based on a handler setting this header, but it doesn't
> matter here since the master's operator / scheduler and agent's
> executor streaming API responses are infinite.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp e074a93666d82944774e7b0c3fa32d7021d93c97 
> 
> 
> Diff: https://reviews.apache.org/r/68553/diff/1/
> 
> 
> Testing
> ---
> 
> Manually verified headers make it through:
> 
> ```
> $ telnet 10.0.49.2 5050
> Trying 10.0.49.2...
> Connected to 10.0.49.2.
> Escape character is '^]'.
> POST /master/api/v1 HTTP/1.1
> Content-Type: application/json
> Accept: application/json
> Content-Length: 20
> 
> {"type":"SUBSCRIBE"}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Content-Type: application/json
> Date: Wed, 29 Aug 2018 01:58:34 GMT
> Connection: close
> 
> 9e
> 154
> {"type":"SUBSCRIBED","subscribed":{"get_state":{"get_tasks":{},"get_executors":{},"get_frameworks":{},"get_agents":{}},"heartbeat_interval_seconds":15.0}}
> 17
> 20
> {"type":"HEARTBEAT"}
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>



Review Request 68553: Set 'Connection: close' in the master's streaming API responses.

2018-08-28 Thread Benjamin Mahler

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68553/
---

Review request for mesos, Gastón Kleiman and Vinod Kone.


Bugs: MESOS-9189
https://issues.apache.org/jira/browse/MESOS-9189


Repository: mesos


Description
---

We've seen some HTTP intermediaries (e.g. ELB) decide to re-use
connections to mesos as an optimization to avoid re-connection
overhead. As a result, when the end-client of the streaming API
disconnects from the intermediary, the intermediary leaves the
connection to Mesos open in an attempt to re-use the connection
for another request once the response completes. Mesos then thinks
that the subscriber never disconnected and the intermediary happily
continues to read the streaming events even though there's no
end-client.

To help indicate to intermediaries that the connection SHOULD NOT
be re-used, we can set the 'Connection: close' header for streaming
API responses. It may not be respected (since the language seems to
be SHOULD NOT), but some intermediaries may respect it and close the
connection if the end-client disconnects.

Note that libprocess' http server currently doesn't close the the
connection based on a handler setting this header, but it doesn't
matter here since the master's operator / scheduler and agent's
executor streaming API responses are infinite.


Diffs
-

  src/master/http.cpp e074a93666d82944774e7b0c3fa32d7021d93c97 


Diff: https://reviews.apache.org/r/68553/diff/1/


Testing
---

Manually verified headers make it through:

```
$ telnet 10.0.49.2 5050
Trying 10.0.49.2...
Connected to 10.0.49.2.
Escape character is '^]'.
POST /master/api/v1 HTTP/1.1
Content-Type: application/json
Accept: application/json
Content-Length: 20

{"type":"SUBSCRIBE"}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/json
Date: Wed, 29 Aug 2018 01:58:34 GMT
Connection: close

9e
154
{"type":"SUBSCRIBED","subscribed":{"get_state":{"get_tasks":{},"get_executors":{},"get_frameworks":{},"get_agents":{}},"heartbeat_interval_seconds":15.0}}
17
20
{"type":"HEARTBEAT"}
```


Thanks,

Benjamin Mahler



Re: Review Request 68068: Added tests for task metadata GC using the default executor.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68068/#review208075
---



PASS: Mesos patch 68068 was successfully built and tested.

Reviews applied: `['68065', '68066', '68067', '68095', '68068']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2255/mesos-review-68068

- Mesos Reviewbot Windows


On Aug. 22, 2018, 6:33 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68068/
> ---
> 
> (Updated Aug. 22, 2018, 6:33 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Qian Zhang.
> 
> 
> Bugs: MESOS-7947
> https://issues.apache.org/jira/browse/MESOS-7947
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This adds two tests that launch a long-lived and short-lived task
> on the same executor.  The tests expect the short-lived task's
> metadata and sandbox to be garbage collected.
> 
> One test restarts the agent before GC to ensure recovery will
> schedule completed tasks for GC too.
> 
> One existing test is modified due to the extra GC event from
> task metadata.
> 
> 
> Diffs
> -
> 
>   src/tests/gc_tests.cpp 4f288cfd70cc53b4064bced96c3c5d6377c1c421 
>   src/tests/slave_recovery_tests.cpp 9710518db30ce3826284f3b7e323bdd9aa207831 
> 
> 
> Diff: https://reviews.apache.org/r/68068/diff/4/
> 
> 
> Testing
> ---
> 
> make check (OSX)
> 
> (Other platforms pending...)
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>



Re: Review Request 68454: Windows: Fetch blobs with V2S2 Docker image manifest.

2018-08-28 Thread Liangyu Zhao via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68454/
---

(Updated Aug. 28, 2018, 11:26 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Gilbert Song, Jie 
Yu, Joseph Wu, and Qian Zhang.


Bugs: MESOS-9159
https://issues.apache.org/jira/browse/MESOS-9159


Repository: mesos


Description
---

DockerFetcher now fetches both V2S1 and V2S2 manifests to save on
disk when agent is running on Windows. Linux part of the code in
agent is unchanged. In addition to fetching from DockerHub,
DockerFetcher now supports fetching from foreign URLs provided in
V2S2 Docker image manifest.


Diffs (updated)
-

  src/uri/fetchers/docker.hpp 7a6193dd75dc10846f52cb039f6a162cc8805c16 
  src/uri/fetchers/docker.cpp 55ca118660872a933a2dc186723bec6a39ee80f7 


Diff: https://reviews.apache.org/r/68454/diff/5/

Changes: https://reviews.apache.org/r/68454/diff/4-5/


Testing
---


Thanks,

Liangyu Zhao



Re: Review Request 68542: De-duplicated identical read-only requests to master.

2018-08-28 Thread Mesos Reviewbot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68542/#review208074
---



Patch looks great!

Reviews applied: [68542]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Aug. 28, 2018, 4:54 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68542/
> ---
> 
> (Updated Aug. 28, 2018, 4:54 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Under certain assumptions, we can safely assume that
> two requests with the same parametesr to the same endpoint
> will return the same result. This is most noticeable for
> the `/state` endpoint.
> 
> In this case, we only need to compute the response once.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
> 
> 
> Diff: https://reviews.apache.org/r/68542/diff/1/
> 
> 
> Testing
> ---
> 
> Accessed `localhost:5050/state` on master including these changes.
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68550: Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68550/#review208073
---



PASS: Mesos patch 68550 was successfully built and tested.

Reviews applied: `['68550']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2254/mesos-review-68550

- Mesos Reviewbot Windows


On Aug. 28, 2018, 9:46 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68550/
> ---
> 
> (Updated Aug. 28, 2018, 9:46 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, Greg Mann, 
> and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.
> 
> 
> Diffs
> -
> 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 997154a8f6c30b7d74ff5e67fb3efa0bc2b94de7 
> 
> 
> Diff: https://reviews.apache.org/r/68550/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 68454: Windows: Fetch blobs with V2S2 Docker image manifest.

2018-08-28 Thread Liangyu Zhao via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68454/
---

(Updated Aug. 28, 2018, 10:38 p.m.)


Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Gilbert Song, Jie 
Yu, Joseph Wu, and Qian Zhang.


Bugs: MESOS-9159
https://issues.apache.org/jira/browse/MESOS-9159


Repository: mesos


Description
---

DockerFetcher now fetches both V2S1 and V2S2 manifests to save on
disk when agent is running on Windows. Linux part of the code in
agent is unchanged. In addition to fetching from DockerHub,
DockerFetcher now supports fetching from foreign URLs provided in
V2S2 Docker image manifest.


Diffs (updated)
-

  src/uri/fetchers/docker.hpp 7a6193dd75dc10846f52cb039f6a162cc8805c16 
  src/uri/fetchers/docker.cpp 55ca118660872a933a2dc186723bec6a39ee80f7 


Diff: https://reviews.apache.org/r/68454/diff/4/

Changes: https://reviews.apache.org/r/68454/diff/3-4/


Testing
---


Thanks,

Liangyu Zhao



Re: Review Request 68550: Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.

2018-08-28 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68550/#review208072
---


Ship it!




Ship It!

- Jie Yu


On Aug. 28, 2018, 9:46 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68550/
> ---
> 
> (Updated Aug. 28, 2018, 9:46 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, Greg Mann, 
> and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.
> 
> 
> Diffs
> -
> 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 997154a8f6c30b7d74ff5e67fb3efa0bc2b94de7 
> 
> 
> Diff: https://reviews.apache.org/r/68550/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>



Re: Review Request 68224: Augmented `Statistics` to work with any collection.

2018-08-28 Thread Alexander Rukletsov


> On Aug. 22, 2018, 2:30 a.m., Benjamin Mahler wrote:
> > 3rdparty/libprocess/include/process/statistics.hpp
> > Line 108 (original), 148 (patched)
> > 
> >
> > Whoops? Can you remove this stray from the diff?

Nope, it participates in type resolution : )


- Alexander


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68224/#review207724
---


On Aug. 14, 2018, 12:48 p.m., Alexander Rukletsov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68224/
> ---
> 
> (Updated Aug. 14, 2018, 12:48 p.m.)
> 
> 
> Review request for mesos, Benno Evers and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> See summary.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/include/process/statistics.hpp 
> e9f1fc23bf83f92a2e7de94dba0df48272cc3394 
>   3rdparty/libprocess/src/tests/statistics_tests.cpp 
> a2a780bf9de018c823b68aa48977fd9fd1b8a064 
> 
> 
> Diff: https://reviews.apache.org/r/68224/diff/3/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Alexander Rukletsov
> 
>



Review Request 68550: Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.

2018-08-28 Thread Chun-Hung Hsiao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68550/
---

Review request for mesos, Alexander Rukletsov, Benjamin Bannier, Greg Mann, and 
Jie Yu.


Repository: mesos


Description
---

Fixed flakiness in the `CreateDestroyDiskRecovery` SLRP test.


Diffs
-

  src/tests/storage_local_resource_provider_tests.cpp 
997154a8f6c30b7d74ff5e67fb3efa0bc2b94de7 


Diff: https://reviews.apache.org/r/68550/diff/1/


Testing
---

make check


Thanks,

Chun-Hung Hsiao



[GitHub] jieyu closed pull request #308: Add Liangyu as contributor.

2018-08-28 Thread GitBox
jieyu closed pull request #308: Add Liangyu as contributor.
URL: https://github.com/apache/mesos/pull/308
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/docs/contributors.yaml b/docs/contributors.yaml
index e58a39a0d0..630ba4b2aa 100755
--- a/docs/contributors.yaml
+++ b/docs/contributors.yaml
@@ -586,6 +586,14 @@
   jira_user: lawrencew
   reviewboard_user: lawrencew
 
+- name: Liangyu Zhao
+  affiliations:
+- {organization: Microsoft}
+  emails:
+- liangyuzhaor...@hotmail.com
+  jira_user: lia...@microsoft.com
+  reviewboard_user: liazha
+
 - name: Lijun Tang
   emails:
 - hercule2...@gmail.com


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


Re: Review Request 68548: Added allocator benchmark test harness.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68548/#review208067
---



PASS: Mesos patch 68548 was successfully built and tested.

Reviews applied: `['68549', '68548']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2253/mesos-review-68548

- Mesos Reviewbot Windows


On Aug. 28, 2018, 12:45 p.m., Kapil Arya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68548/
> ---
> 
> (Updated Aug. 28, 2018, 12:45 p.m.)
> 
> 
> Review request for mesos, Meng Zhu and Till Toenshoff.
> 
> 
> Bugs: MESOS-9187
> https://issues.apache.org/jira/browse/MESOS-9187
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The new harness allows us to run allocators with different framework and
> agent profiles.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am cf0cf22b3f582d3d4427f13288af3845aef45263 
>   src/tests/hierarchical_allocator_benchmarks.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68548/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kapil Arya
> 
>



[GitHub] liangyuRain opened a new pull request #308: Add Liangyu as contributor.

2018-08-28 Thread GitBox
liangyuRain opened a new pull request #308: Add Liangyu as contributor.
URL: https://github.com/apache/mesos/pull/308
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


Re: Review Request 68454: Windows: Fetch blobs with V2S2 Docker image manifest.

2018-08-28 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68454/#review208066
---




src/uri/fetchers/docker.cpp
Lines 100 (patched)


Looks like you did this because we use `path::join` to construct URL. I 
think this can be avoided if we just use `strings::join` when constructing url. 
Constructing url using `path::join` does not make sense.



src/uri/fetchers/docker.cpp
Lines 935-937 (patched)


No need for this tmp variable.



src/uri/fetchers/docker.cpp
Lines 982-986 (patched)


Ditto. No need for this tmp variable.


- Jie Yu


On Aug. 25, 2018, 9:57 p.m., Liangyu Zhao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68454/
> ---
> 
> (Updated Aug. 25, 2018, 9:57 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Andrew Schwartzmeyer, Gilbert Song, 
> Jie Yu, Joseph Wu, and Qian Zhang.
> 
> 
> Bugs: MESOS-9159
> https://issues.apache.org/jira/browse/MESOS-9159
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> DockerFetcher now fetches both V2S1 and V2S2 manifests to save on
> disk when agent is running on Windows. Linux part of the code in
> agent is unchanged. In addition to fetching from DockerHub,
> DockerFetcher now supports fetching from foreign URLs provided in
> V2S2 Docker image manifest.
> 
> 
> Diffs
> -
> 
>   src/uri/fetchers/docker.hpp 7a6193dd75dc10846f52cb039f6a162cc8805c16 
>   src/uri/fetchers/docker.cpp 55ca118660872a933a2dc186723bec6a39ee80f7 
> 
> 
> Diff: https://reviews.apache.org/r/68454/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Liangyu Zhao
> 
>



Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Mesos Reviewbot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/#review208065
---



Patch looks great!

Reviews applied: [68321, 68440, 68441, 68442, 68473, 68537]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Aug. 28, 2018, 2:02 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68537/
> ---
> 
> (Updated Aug. 28, 2018, 2:02 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This commit fixes several minor style issues:
>  - Sorted member function declarations of `ReadOnlyHandler`
>alphabetically.
>  - Added notes to remind readers of the fact that requests
>to certain endpoints are batched.
>  - Changed captured variable in `/frameworks` endpoint handler.
> 
> 
> Diffs
> -
> 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
>   src/master/readonly_handler.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68537/diff/2/
> 
> 
> Testing
> ---
> 
> Started Internal CI Run (Jenkins Id #4264)
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68355: Added a CNI test for networking statistics.

2018-08-28 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68355/#review208062
---


Ship it!




Ship It!

- Jie Yu


On Aug. 17, 2018, 3:37 a.m., Sergey Urbanovich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68355/
> ---
> 
> (Updated Aug. 17, 2018, 3:37 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Qian Zhang.
> 
> 
> Bugs: MESOS-5647
> https://issues.apache.org/jira/browse/MESOS-5647
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This is a veth CNI plugin that is written in bash. It creates a veth
> virtual network pair, one end of the pair will be moved to container
> network namespace.
> 
> The veth CNI plugin uses 203.0.113.0/24 subnet, it is reserved for
> documentation and examples [rfc5737]. The plugin can allocate up to
> 128 veth pairs.
> 
> 
> Diffs
> -
> 
>   src/tests/containerizer/cni_isolator_tests.cpp 
> 63b109bdbc23733aa3d4cdce1de5385ad9933be8 
>   src/tests/environment.cpp 8c6ec5854268d3c497b9671a95768d6a174673c6 
> 
> 
> Diff: https://reviews.apache.org/r/68355/diff/6/
> 
> 
> Testing
> ---
> 
> bin/mesos-tests.sh --verbose 
> --gtest_filter="CniIsolatorTest.ROOT_VerifyResourceStatistics" 
> --gtest_break_on_failure --gtest_repeat=100
> 
> 
> Thanks,
> 
> Sergey Urbanovich
> 
>



Re: Review Request 68442: Added '/frameworks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68442/
---

(Updated Aug. 28, 2018, 8:10 p.m.)


Review request for mesos and Alexander Rukletsov.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/frameworks' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68442/diff/2/


Testing (updated)
---

See https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68441: Added '/slaves' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68441/
---

(Updated Aug. 28, 2018, 8:08 p.m.)


Review request for mesos and Alexander Rukletsov.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/slaves' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68441/diff/2/


Testing
---

See https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68440: Added '/tasks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68440/
---

(Updated Aug. 28, 2018, 8:08 p.m.)


Review request for mesos and Alexander Rukletsov.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/tasks' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68440/diff/2/


Testing
---

See https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68442: Added '/frameworks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68442/
---

(Updated Aug. 28, 2018, 8:08 p.m.)


Review request for mesos and Alexander Rukletsov.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/frameworks' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68442/diff/2/


Testing
---


Thanks,

Benno Evers



Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/#review208060
---


Ship it!




Ship It!

- Alexander Rukletsov


On Aug. 28, 2018, 2:02 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68537/
> ---
> 
> (Updated Aug. 28, 2018, 2:02 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This commit fixes several minor style issues:
>  - Sorted member function declarations of `ReadOnlyHandler`
>alphabetically.
>  - Added notes to remind readers of the fact that requests
>to certain endpoints are batched.
>  - Changed captured variable in `/frameworks` endpoint handler.
> 
> 
> Diffs
> -
> 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
>   src/master/readonly_handler.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68537/diff/2/
> 
> 
> Testing
> ---
> 
> Started Internal CI Run (Jenkins Id #4264)
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Review Request 68549: Added pause and resume helpers to the allocator.

2018-08-28 Thread Kapil Arya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68549/
---

Review request for mesos, Meng Zhu and Till Toenshoff.


Bugs: MESOS-9187
https://issues.apache.org/jira/browse/MESOS-9187


Repository: mesos


Description
---

Currently these helpers are only used in the allocator
tests to prevent event-driven allocations that triggered
when adding frameworks and agents.

This is the same as Meng's patch in https://reviews.apache.org/r/68049/ with a 
compilation fix.


Diffs
-

  include/mesos/allocator/allocator.hpp 
fadd4e8e32c9edca4fefa1cd5f8c8720188d3ca5 
  src/master/allocator/mesos/allocator.hpp 
e65a9698ffe514b5f129417694190790712de508 
  src/master/allocator/mesos/hierarchical.hpp 
e3adb5dbb968684f6f63477f56a6abf4f288c896 
  src/tests/allocator.hpp 1d844464741b3f40890198e2503593c76de870df 


Diff: https://reviews.apache.org/r/68549/diff/1/


Testing
---


Thanks,

Kapil Arya



Review Request 68548: Added allocator benchmark test harness.

2018-08-28 Thread Kapil Arya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68548/
---

Review request for mesos, Meng Zhu and Till Toenshoff.


Bugs: MESOS-9187
https://issues.apache.org/jira/browse/MESOS-9187


Repository: mesos


Description
---

The new harness allows us to run allocators with different framework and
agent profiles.


Diffs
-

  src/Makefile.am cf0cf22b3f582d3d4427f13288af3845aef45263 
  src/tests/hierarchical_allocator_benchmarks.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/68548/diff/1/


Testing
---


Thanks,

Kapil Arya



Re: Review Request 68473: Moved members of `ReadOnlyHandler` into separate file.

2018-08-28 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68473/#review208059
---


Fix it, then Ship it!





src/master/readonly_handler.cpp
Lines 19-20 (patched)


No, darling : )



src/master/readonly_handler.cpp
Lines 26-27 (patched)


Nope : )


- Alexander Rukletsov


On Aug. 27, 2018, 5:44 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68473/
> ---
> 
> (Updated Aug. 27, 2018, 5:44 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Moved the member functions of class `ReadOnlyHandler` into
> the new file `readonly_handler.cpp`. This follows the pattern
> established by `weights_handler.cpp` and `quota_handler.cpp`.
> 
> As part of this move, it was also necessary to move some JSON
> serialization that are used from both `master.cpp` and
> `readonly_handler.cpp` to a new pair of files `json.{cpp,hpp}`
> that can be used from both places.
> 
> 
> Diffs
> -
> 
>   src/CMakeLists.txt a80b011bbd5c418ae66eb8dd7697d070168462bb 
>   src/Makefile.am cf0cf22b3f582d3d4427f13288af3845aef45263 
>   src/common/http.hpp 19f0da2004391af7a526026c5e293fb01c01e97c 
>   src/common/http.cpp 932111dde6e49c697bb23e322cae934effacd1b1 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/readonly_handler.cpp PRE-CREATION 
>   src/slave/http.cpp 1b6d266a1a7821c9de6871cbca43317b3c392a32 
> 
> 
> Diff: https://reviews.apache.org/r/68473/diff/4/
> 
> 
> Testing
> ---
> 
> `make check`
> Internal CI run (Build #4249)
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68440: Added '/tasks' to the set of batched master endpoints.

2018-08-28 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68440/#review208058
---


Fix it, then Ship it!





src/master/http.cpp
Line 4043 (original), 4043 (patched)


Extra blank line


- Alexander Rukletsov


On Aug. 28, 2018, 10:28 a.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68440/
> ---
> 
> (Updated Aug. 28, 2018, 10:28 a.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added '/tasks' to the set of batched master endpoints.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
> 
> 
> Diff: https://reviews.apache.org/r/68440/diff/2/
> 
> 
> Testing
> ---
> 
> See https://reviews.apache.org/r/68473/
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68543: Added stout helper to parse strings to protobuf messages.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68543/#review208057
---



PASS: Mesos patch 68543 was successfully built and tested.

Reviews applied: `['68543']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2252/mesos-review-68543

- Mesos Reviewbot Windows


On Aug. 28, 2018, 5:25 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68543/
> ---
> 
> (Updated Aug. 28, 2018, 5:25 p.m.)
> 
> 
> Review request for mesos, Joseph Wu and Kevin Klues.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added stout helper to parse strings to protobuf messages.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/protobuf.hpp 
> 1d03e1e3a8dd642f7239d777fb04759caf100a8b 
>   3rdparty/stout/tests/protobuf_tests.cpp 
> 95cdc67cdab0aeef2ce18aa0c99bc2952c2b5dc5 
> 
> 
> Diff: https://reviews.apache.org/r/68543/diff/1/
> 
> 
> Testing
> ---
> 
> New test passes.
> 
> 
> Thanks,
> 
> Greg Mann
> 
>



Re: Review Request 68543: Added stout helper to parse strings to protobuf messages.

2018-08-28 Thread Joseph Wu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68543/#review208056
---




3rdparty/stout/include/stout/protobuf.hpp
Lines 778 (patched)


Using `flags::parse` will include some deprecated behavior (if the string 
starts with `"/"`, we will attempt to read a location on disk).  
`JSON::parse` would be a better choice.


- Joseph Wu


On Aug. 28, 2018, 10:25 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68543/
> ---
> 
> (Updated Aug. 28, 2018, 10:25 a.m.)
> 
> 
> Review request for mesos, Joseph Wu and Kevin Klues.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added stout helper to parse strings to protobuf messages.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/protobuf.hpp 
> 1d03e1e3a8dd642f7239d777fb04759caf100a8b 
>   3rdparty/stout/tests/protobuf_tests.cpp 
> 95cdc67cdab0aeef2ce18aa0c99bc2952c2b5dc5 
> 
> 
> Diff: https://reviews.apache.org/r/68543/diff/1/
> 
> 
> Testing
> ---
> 
> New test passes.
> 
> 
> Thanks,
> 
> Greg Mann
> 
>



Re: Review Request 68538: Added Python 3.6 and and pip to Docker images.

2018-08-28 Thread Mesos Reviewbot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68538/#review208055
---



Patch looks great!

Reviews applied: [68538]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Aug. 28, 2018, 2:17 p.m., Robin Gögge wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68538/
> ---
> 
> (Updated Aug. 28, 2018, 2:17 p.m.)
> 
> 
> Review request for mesos, Armand Grillet and Kevin Klues.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Following the update of the CLI to Python 3, we embed Python 3.6
> (the minimum required Python version) to the docker images used
> during continuous integration.
> 
> 
> Diffs
> -
> 
>   support/mesos-build/centos-7.dockerfile 
> 068f946f8410772afd9aa45c6f864e475efe84c9 
>   support/mesos-build/ubuntu-16.04-arm.dockerfile 
> 352156fb14d90a4b248bc5d15f1d0127bec00161 
>   support/mesos-build/ubuntu-16.04.dockerfile 
> 503b2e370b9222a0e92b8d5db2b08256df3adef8 
> 
> 
> Diff: https://reviews.apache.org/r/68538/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robin Gögge
> 
>



Re: Review Request 68420: Added /files API test for reserved query characters.

2018-08-28 Thread Andrew Schwartzmeyer


> On Aug. 23, 2018, 1:02 p.m., Benjamin Mahler wrote:
> > src/tests/files_tests.cpp
> > Lines 331-332 (patched)
> > 
> >
> > This is a little odd to read without the context, this tests reserved 
> > characters by using `%`, but we could use anything other reserved character 
> > to acheive the same test, e.g. `+` seems the simplest:
> > 
> > ```
> >   // We use a reserved character for query parameters `+` to
> >   // ensure it is encoded and decoded correctly.
> >   const string filename = "foo+bar";
> > ```
> > 
> > If you'd like to stick with using `%`, probably we should just say:
> > 
> > ```
> >   // We use a reserved character for query parameters `%` to
> >   // ensure it is encoded and decoded correctly.
> >   const string filename = "foo%3Abar";
> > ```
> > 
> > And not do line 334 as it's irrelevant to this test
> 
> Andrew Schwartzmeyer wrote:
> You couldn't use `+` because it wouldn't have the same problem: the 
> second decode would not turn `+` into something else (this is why we haven't 
> run into this problem before with other characters, even a literal `:`). The 
> problem is that `%3A` _can_ be decoded, and so decoding more times than 
> encoding causes it to erroneously become `:`. The number of encodes and 
> decodes must be balanced.
> 
> Also, line 334 is relevant to this test: it's demonstrating that the 
> query can be erroneously decoded too many times.

I explained why thits is explicitly an entire percent-encoding, and am 
considering this fixed. Thanks!


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68420/#review207830
---


On Aug. 28, 2018, 10:40 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68420/
> ---
> 
> (Updated Aug. 28, 2018, 10:40 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-9168
> https://issues.apache.org/jira/browse/MESOS-9168
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Due to a bug in the HTTP client in libprocess (MESOS-9168), the
> use of reserved query characters in paths did not work when using
> the HTTP client in libprocess. This adds a test to ensure that
> the /files API correctly handles reserved query characters in
> file paths.
> 
> 
> Diffs
> -
> 
>   src/tests/files_tests.cpp d09279077ec704167991933349ee943ab44d0aa9 
> 
> 
> Diff: https://reviews.apache.org/r/68420/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>



Re: Review Request 68420: Added /files API test for reserved query characters.

2018-08-28 Thread Andrew Schwartzmeyer

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68420/
---

(Updated Aug. 28, 2018, 10:40 a.m.)


Review request for mesos and Benjamin Mahler.


Changes
---

Updated with better explanation and a new test name and commit message.


Summary (updated)
-

Added /files API test for reserved query characters.


Bugs: MESOS-9168
https://issues.apache.org/jira/browse/MESOS-9168


Repository: mesos


Description (updated)
---

Due to a bug in the HTTP client in libprocess (MESOS-9168), the
use of reserved query characters in paths did not work when using
the HTTP client in libprocess. This adds a test to ensure that
the /files API correctly handles reserved query characters in
file paths.


Diffs (updated)
-

  src/tests/files_tests.cpp d09279077ec704167991933349ee943ab44d0aa9 


Diff: https://reviews.apache.org/r/68420/diff/2/

Changes: https://reviews.apache.org/r/68420/diff/1-2/


Testing
---


Thanks,

Andrew Schwartzmeyer



Review Request 68543: Added stout helper to parse strings to protobuf messages.

2018-08-28 Thread Greg Mann

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68543/
---

Review request for mesos, Joseph Wu and Kevin Klues.


Repository: mesos


Description
---

Added stout helper to parse strings to protobuf messages.


Diffs
-

  3rdparty/stout/include/stout/protobuf.hpp 
1d03e1e3a8dd642f7239d777fb04759caf100a8b 
  3rdparty/stout/tests/protobuf_tests.cpp 
95cdc67cdab0aeef2ce18aa0c99bc2952c2b5dc5 


Diff: https://reviews.apache.org/r/68543/diff/1/


Testing
---

New test passes.


Thanks,

Greg Mann



Re: Review Request 68542: De-duplicated identical read-only requests to master.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68542/#review208052
---



FAIL: Failed to apply the current review.

Failed command: `python.exe .\support\python3\apply-reviews.py -n -r 68542`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2251/mesos-review-68542

Relevant logs:

- 
[apply-review-68542.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2251/mesos-review-68542/logs/apply-review-68542.log):

```
error: patch failed: src/master/http.cpp:2383
error: src/master/http.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Aug. 28, 2018, 4:54 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68542/
> ---
> 
> (Updated Aug. 28, 2018, 4:54 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Under certain assumptions, we can safely assume that
> two requests with the same parametesr to the same endpoint
> will return the same result. This is most noticeable for
> the `/state` endpoint.
> 
> In this case, we only need to compute the response once.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
> 
> 
> Diff: https://reviews.apache.org/r/68542/diff/1/
> 
> 
> Testing
> ---
> 
> Accessed `localhost:5050/state` on master including these changes.
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68538: Added Python 3.6 and and pip to Docker images.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68538/#review208051
---



PASS: Mesos patch 68538 was successfully built and tested.

Reviews applied: `['68538']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2250/mesos-review-68538

- Mesos Reviewbot Windows


On Aug. 28, 2018, 2:17 p.m., Robin Gögge wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68538/
> ---
> 
> (Updated Aug. 28, 2018, 2:17 p.m.)
> 
> 
> Review request for mesos, Armand Grillet and Kevin Klues.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Following the update of the CLI to Python 3, we embed Python 3.6
> (the minimum required Python version) to the docker images used
> during continuous integration.
> 
> 
> Diffs
> -
> 
>   support/mesos-build/centos-7.dockerfile 
> 068f946f8410772afd9aa45c6f864e475efe84c9 
>   support/mesos-build/ubuntu-16.04-arm.dockerfile 
> 352156fb14d90a4b248bc5d15f1d0127bec00161 
>   support/mesos-build/ubuntu-16.04.dockerfile 
> 503b2e370b9222a0e92b8d5db2b08256df3adef8 
> 
> 
> Diff: https://reviews.apache.org/r/68538/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robin Gögge
> 
>



Review Request 68542: De-duplicated identical read-only requests to master.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68542/
---

Review request for mesos and Alexander Rukletsov.


Repository: mesos


Description
---

Under certain assumptions, we can safely assume that
two requests with the same parametesr to the same endpoint
will return the same result. This is most noticeable for
the `/state` endpoint.

In this case, we only need to compute the response once.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68542/diff/1/


Testing
---

Accessed `localhost:5050/state` on master including these changes.


Thanks,

Benno Evers



Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/#review208046
---



PASS: Mesos patch 68537 was successfully built and tested.

Reviews applied: `['68321', '68440', '68441', '68442', '68473', '68537']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2249/mesos-review-68537

- Mesos Reviewbot Windows


On Aug. 28, 2018, 2:02 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68537/
> ---
> 
> (Updated Aug. 28, 2018, 2:02 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This commit fixes several minor style issues:
>  - Sorted member function declarations of `ReadOnlyHandler`
>alphabetically.
>  - Added notes to remind readers of the fact that requests
>to certain endpoints are batched.
>  - Changed captured variable in `/frameworks` endpoint handler.
> 
> 
> Diffs
> -
> 
>   src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
>   src/master/readonly_handler.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68537/diff/2/
> 
> 
> Testing
> ---
> 
> Started Internal CI Run (Jenkins Id #4264)
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68535: Updated Python dependencies for 3.7.

2018-08-28 Thread Benjamin Bannier

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68535/#review208041
---




support/pip-requirements.txt
Line 2 (original), 2 (patched)


This does not work for me when using a python2 setup,

$ ./support/apply-reviews.py -r 68535 -c -n
Congratulations! You have Python 3 installed correctly.
Please start using the scripts in `support/python3`.
Please also set the environment variable `MESOS_SUPPORT_PYTHON` to `3`
so that the Git hooks use the Python 3 scripts.
2018-08-28 16:52:32 URL:https://reviews.apache.org/r/68535/diff/raw/ 
[2156/2156] -> "68535.patch" [1]
The "pip-requirements.txt" file has changed.
Rebuilding virtualenv...
  Could not find a version that satisfies the requirement pylint==2.1.1 
(from -r /Users/bbannier/src/mesos/support/pip-requirements.txt (line 2)) (from 
versions: 0.15.2, 0.16.0, 0.18.0, 0.18.1, 0.19.0, 0.20.0, 0.21.0, 0.21.1, 
0.21.2, 0.21.3, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.25.1, 0.25.2, 0.26.0, 0.27.0, 
0.28.0, 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 
1.4.4, 1.4.5, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.6.0, 1.6.1, 
1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.4, 1.7.5, 1.7.6, 
1.8.0, 1.8.1, 1.8.2, 1.8.3, 1.8.4, 1.9.0, 1.9.1, 1.9.2, 1.9.3)
No matching distribution found for pylint==2.1.1 (from -r 
/Users/bbannier/src/mesos/support/pip-requirements.txt (line 2))
New python executable in 
/Users/bbannier/src/mesos/support/.virtualenv/bin/python2.7
Also creating executable in 
/Users/bbannier/src/mesos/support/.virtualenv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/local/opt/python@2/bin/python
Requirement already up-to-date: pip in 
./support/.virtualenv/lib/python2.7/site-packages (18.0)
Collecting nodeenv==1.3.2 (from -r 
/Users/bbannier/src/mesos/support/pip-requirements.txt (line 1))
  Using cached 
https://files.pythonhosted.org/packages/36/ca/1a635c6cd6049105fadfdc05c8de364f368fcc7aebad3081dd8ccb0c5c7f/nodeenv-1.3.2.tar.gz
Collecting pylint==2.1.1 (from -r 
/Users/bbannier/src/mesos/support/pip-requirements.txt (line 2))

Let's either pin a version which works for both python2.7 and >=python3.6, 
or introduce a dedicated virtualenv for the files in `support/python3`.


- Benjamin Bannier


On Aug. 28, 2018, 10:05 a.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68535/
> ---
> 
> (Updated Aug. 28, 2018, 10:05 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Robin Gögge.
> 
> 
> Bugs: MESOS-9186
> https://issues.apache.org/jira/browse/MESOS-9186
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> All the dependencies are now fixed so that we will only need to update
> when a new Python version is being released.
> 
> 
> Diffs
> -
> 
>   src/python/cli_new/pip-requirements.txt 
> 8645912a63b22878f477e96bfa8bc95abe893f85 
>   src/python/cli_new/tox.ini a9a2e69156484a4d172a2746f8cc51089445ae17 
>   src/python/lib/tox.ini 0779fb22d1e8af2dbd17d0fb6b1c8cf3dc2e37d3 
>   support/pip-requirements.txt 89c2de287bc26bbde3006b587a4a8c1bae6f600b 
> 
> 
> Diff: https://reviews.apache.org/r/68535/diff/1/
> 
> 
> Testing
> ---
> 
> Used the scripts and saw that the error describe in MESOS-9186 is gone. 
> Multiple packages had issues with Python 3.7 and I have used versions that 
> include fixes for those.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 68234: Added `AgentAPITest.LaunchNestedContainerWithUnknownParent` test.

2018-08-28 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68234/#review208039
---


Ship it!




Ship It!

- Alexander Rukletsov


On Aug. 27, 2018, 2:57 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68234/
> ---
> 
> (Updated Aug. 27, 2018, 2:57 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Gilbert Song, and Qian Zhang.
> 
> 
> Bugs: MESOS-9185
> https://issues.apache.org/jira/browse/MESOS-9185
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test verifies that launch nested container fails when the parent
> container is unknown to the containerizer.
> 
> 
> Diffs
> -
> 
>   src/tests/api_tests.cpp ee82350f7a6c6d44ba0590608e7d9a223c0be169 
> 
> 
> Diff: https://reviews.apache.org/r/68234/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>



Re: Review Request 68235: Cleaned up container on launch failures in composing containerizer.

2018-08-28 Thread Alexander Rukletsov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68235/#review208040
---


Ship it!




Ship It!

- Alexander Rukletsov


On Aug. 27, 2018, 2:57 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68235/
> ---
> 
> (Updated Aug. 27, 2018, 2:57 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Gilbert Song, and Qian Zhang.
> 
> 
> Bugs: MESOS-9185
> https://issues.apache.org/jira/browse/MESOS-9185
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously, if a parent container was unknown to the composing
> containerizer during an attempt to launch a nested container
> via `ComposingContainerizerProcess::launch()`, the composing
> containerizer returned an error without cleaning up the container.
> The `containerizer` field was uninitialized, so a further attempt
> to remove or destroy the nested container led to segfault.
> 
> This patch removes the container when the parent container is unknown.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/composing.cpp 
> 521645839b422d455da08ce7ab306f83da0b88ad 
> 
> 
> Diff: https://reviews.apache.org/r/68235/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check (Fedora 25)
> 
> This test fixes agent's segfault when running 
> `AgentAPITest.LaunchNestedContainerWithUnknownParent` test, which can be 
> found in the previous patch.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>



Review Request 68236: Added Seccomp-related protobuf messages.

2018-08-28 Thread Andrei Budnik

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68236/
---

Review request for mesos.


Repository: mesos


Description
---

See summary.


Testing
---


Thanks,

Andrei Budnik



Re: Review Request 68495: Made command check always waits before removing the nested container.

2018-08-28 Thread Qian Zhang


> On Aug. 28, 2018, 1:16 a.m., Alexander Rukletsov wrote:
> > src/checks/checker_process.cpp
> > Lines 878-889 (original), 890-901 (patched)
> > 
> >
> > It looks like we should always call `waitNestedContainer()` after we 
> > said `previousCheckContainerId = checkContainerId;`. For example here.
> > 
> > Maybe it makes sense to call `waitNestedContainer()` right in the 
> > beginning? We can end up calling it twice, but I think it's fine?
> 
> Qian Zhang wrote:
> > Maybe it makes sense to call waitNestedContainer() right in the 
> beginning? We can end up calling it twice, but I think it's fine?
> 
> That means we will call agent API `WAIT_NESTED_CONTAINER` twice for each 
> successful launch of check container, I think that might be a burden for 
> agent in a large scale env. So I'd still prefer to call it only in the places 
> where we have to do it.

And for the case (L879:L888, i.e., the connection to agent failed) that you 
pointed out above, I think when the connection to agent is back (e.g., agent 
starts up again), the check container will be treated as orphan container and 
destroyed by agent, and then we will remove it here: 
https://github.com/apache/mesos/blob/1.6.1/src/checks/checker_process.cpp#L616:L638.
 However I am going to post another patch to change these codes 
(https://github.com/apache/mesos/blob/1.6.1/src/checks/checker_process.cpp#L660:L664)
 to something like:
```
  promise->discard();
} else {
  previousCheckContainerId = None();
  _nestedCommandCheck(promise, cmd, nested);
}
```
In this way, if we fail to remove the check container (e.g., due to agent has 
not finished recovery, or the check container is still in `DESTROYING` state), 
we will try to remove it again.


- Qian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68495/#review207984
---


On Aug. 24, 2018, 5:54 p.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68495/
> ---
> 
> (Updated Aug. 24, 2018, 5:54 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Alexander Rukletsov, Gastón Kleiman, 
> and Gilbert Song.
> 
> 
> Bugs: MESOS-8568
> https://issues.apache.org/jira/browse/MESOS-8568
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made command check always waits before removing the nested container.
> 
> 
> Diffs
> -
> 
>   src/checks/checker_process.cpp 77a76f465fe57eab89f027b5acb74c2339551678 
> 
> 
> Diff: https://reviews.apache.org/r/68495/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>



Review Request 68538: Added Python 3.6 and and pip to Docker images.

2018-08-28 Thread Robin Gögge

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68538/
---

Review request for mesos, Armand Grillet and Kevin Klues.


Repository: mesos


Description
---

Following the update of the CLI to Python 3, we embed Python 3.6
(the minimum required Python version) to the docker images used
during continuous integration.


Diffs
-

  support/mesos-build/centos-7.dockerfile 
068f946f8410772afd9aa45c6f864e475efe84c9 
  support/mesos-build/ubuntu-16.04-arm.dockerfile 
352156fb14d90a4b248bc5d15f1d0127bec00161 
  support/mesos-build/ubuntu-16.04.dockerfile 
503b2e370b9222a0e92b8d5db2b08256df3adef8 


Diff: https://reviews.apache.org/r/68538/diff/1/


Testing
---


Thanks,

Robin Gögge



Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/
---

(Updated Aug. 28, 2018, 2:02 p.m.)


Review request for mesos and Alexander Rukletsov.


Changes
---

Fixed lambda capture list.


Repository: mesos


Description
---

This commit fixes several minor style issues:
 - Sorted member function declarations of `ReadOnlyHandler`
   alphabetically.
 - Added notes to remind readers of the fact that requests
   to certain endpoints are batched.
 - Changed captured variable in `/frameworks` endpoint handler.


Diffs (updated)
-

  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
  src/master/readonly_handler.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/68537/diff/2/

Changes: https://reviews.apache.org/r/68537/diff/1-2/


Testing
---

Started Internal CI Run (Jenkins Id #4262)


Thanks,

Benno Evers



Re: Review Request 68535: Updated Python dependencies for 3.7.

2018-08-28 Thread Robin Gögge

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68535/#review208016
---



LGTM. Built Mesos with `PYTHON_3=python3 ../configure --enable-new-cli 
--disable-java --disable-werror`, python3 being python 3.7 and it worked 
correctly. This wasn't the case before, see stacktrace in 
https://issues.apache.org/jira/browse/MESOS-9186.

- Robin Gögge


On Aug. 28, 2018, 8:05 a.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68535/
> ---
> 
> (Updated Aug. 28, 2018, 8:05 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Robin Gögge.
> 
> 
> Bugs: MESOS-9186
> https://issues.apache.org/jira/browse/MESOS-9186
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> All the dependencies are now fixed so that we will only need to update
> when a new Python version is being released.
> 
> 
> Diffs
> -
> 
>   src/python/cli_new/pip-requirements.txt 
> 8645912a63b22878f477e96bfa8bc95abe893f85 
>   src/python/cli_new/tox.ini a9a2e69156484a4d172a2746f8cc51089445ae17 
>   src/python/lib/tox.ini 0779fb22d1e8af2dbd17d0fb6b1c8cf3dc2e37d3 
>   support/pip-requirements.txt 89c2de287bc26bbde3006b587a4a8c1bae6f600b 
> 
> 
> Diff: https://reviews.apache.org/r/68535/diff/1/
> 
> 
> Testing
> ---
> 
> Used the scripts and saw that the error describe in MESOS-9186 is gone. 
> Multiple packages had issues with Python 3.7 and I have used versions that 
> include fixes for those.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/#review208027
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['68321', '68440', '68441', '68442', '68473', '68537']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2248/mesos-review-68537

Relevant logs:

- 
[mesos-tests-cmake.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2248/mesos-review-68537/logs/mesos-tests-cmake.log):

```
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\recordio.c(170):
 warning C4267: '=': conversion from 'size_t' to 'int32_t', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\zookeeper.vcxproj]
 [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\mt_adaptor.c(496):
 warning C4244: '=': conversion from 'time_t' to 'int32_t', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\zookeeper.vcxproj]
 [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\winport.c(256):
 warning C4090: 'function': different 'const' qualifiers 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\zookeeper.vcxproj]
 [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\winport.c(166):
 warning C4716: 'pthread_cond_broadcast': must return a value 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\zookeeper.vcxproj]
 [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\winport.c(205):
 warning C4716: 'pthread_cond_wait': must return a value 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\zookeeper.vcxproj]
 [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(124):
 warning C4996: 'fopen': This function or variable may be unsafe. Consider 
using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See 
online help for details. 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(128):
 warning C4267: 'initializing': conversion from 'size_t' to 'int', possible 
loss of data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(279):
 warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(301):
 warning C4267: 'initializing': conversion from 'size_t' to 'int', possible 
loss of data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(368):
 warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(372):
 warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(512):
 warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of 
data 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(543):
 warning C4996: 'strcpy': This function or variable may be unsafe. Consider 
using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. 
See online help for details. 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] 
[D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
 
d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(548):
 warning 

Re: Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/
---

(Updated Aug. 28, 2018, 10:51 a.m.)


Review request for mesos and Alexander Rukletsov.


Changes
---

Updated 'testing done' section.


Repository: mesos


Description
---

This commit fixes several minor style issues:
 - Sorted member function declarations of `ReadOnlyHandler`
   alphabetically.
 - Added notes to remind readers of the fact that requests
   to certain endpoints are batched.
 - Changed captured variable in `/frameworks` endpoint handler.


Diffs
-

  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
  src/master/readonly_handler.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/68537/diff/1/


Testing (updated)
---

Started Internal CI Run (Jenkins Id #4262)


Thanks,

Benno Evers



Re: Review Request 68442: Added '/frameworks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Line 1690 (original), 1700 (patched)
> > 
> >
> > This seems inconsistent to other handlers you've introduced: you've 
> > been capturing `master` there.

Fixed in https://reviews.apache.org/r/68537/


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Line 1734 (original), 1744 (patched)
> > 
> >
> > `std::move`

Dropping this since we agreed in offline discussions that it would not make a 
difference.


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/master.hpp
> > Lines 1420-1424 (patched)
> > 
> >
> > let's keep 'em sorted

Fixed in https://reviews.apache.org/r/68537/


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/master.hpp
> > Line 1464 (original), 1469 (patched)
> > 
> >
> > // NOTE: Requests to this endpoint are batched.
> > 
> > or grouping, see https://reviews.apache.org/r/68441/

Fixed in https://reviews.apache.org/r/68537/


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68442/#review207961
---


On Aug. 22, 2018, 1:16 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68442/
> ---
> 
> (Updated Aug. 22, 2018, 1:16 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added '/frameworks' to the set of batched master endpoints.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 
> 
> 
> Diff: https://reviews.apache.org/r/68442/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68441: Added '/slaves' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Lines 2525 (patched)
> > 
> >
> > Let's fix this inconsistency and pass `request.url.query.get("jsonp")` 
> > directly like you do with all other handlers.

Fixed in https://reviews.apache.org/r/68537/


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Lines 2528 (patched)
> > 
> >
> > `return std::move(OK(...));`

Dropping this since we agreed in offline discussions that it would not make a 
difference.


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/master.hpp
> > Lines 1491-1492 (original), 1496-1497 (patched)
> > 
> >
> > // NOTE: Requests to this endpoint are batched.
> > 
> > Or maybe even better: group batched endpoints together.

Fixed in https://reviews.apache.org/r/68537/


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68441/#review207960
---


On Aug. 22, 2018, 1:16 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68441/
> ---
> 
> (Updated Aug. 22, 2018, 1:16 p.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added '/slaves' to the set of batched master endpoints.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 
> 
> 
> Diff: https://reviews.apache.org/r/68441/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Review Request 68537: Cleaned up some style issues in `ReadOnlyHandler`.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68537/
---

Review request for mesos and Alexander Rukletsov.


Repository: mesos


Description
---

This commit fixes several minor style issues:
 - Sorted member function declarations of `ReadOnlyHandler`
   alphabetically.
 - Added notes to remind readers of the fact that requests
   to certain endpoints are batched.
 - Changed captured variable in `/frameworks` endpoint handler.


Diffs
-

  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 
  src/master/readonly_handler.cpp PRE-CREATION 


Diff: https://reviews.apache.org/r/68537/diff/1/


Testing
---


Thanks,

Benno Evers



Re: Review Request 68535: Updated Python dependencies for 3.7.

2018-08-28 Thread Mesos Reviewbot

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68535/#review208022
---



Patch looks great!

Reviews applied: [68535]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Aug. 28, 2018, 8:05 a.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68535/
> ---
> 
> (Updated Aug. 28, 2018, 8:05 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Robin Gögge.
> 
> 
> Bugs: MESOS-9186
> https://issues.apache.org/jira/browse/MESOS-9186
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> All the dependencies are now fixed so that we will only need to update
> when a new Python version is being released.
> 
> 
> Diffs
> -
> 
>   src/python/cli_new/pip-requirements.txt 
> 8645912a63b22878f477e96bfa8bc95abe893f85 
>   src/python/cli_new/tox.ini a9a2e69156484a4d172a2746f8cc51089445ae17 
>   src/python/lib/tox.ini 0779fb22d1e8af2dbd17d0fb6b1c8cf3dc2e37d3 
>   support/pip-requirements.txt 89c2de287bc26bbde3006b587a4a8c1bae6f600b 
> 
> 
> Diff: https://reviews.apache.org/r/68535/diff/1/
> 
> 
> Testing
> ---
> 
> Used the scripts and saw that the error describe in MESOS-9186 is gone. 
> Multiple packages had issues with Python 3.7 and I have used versions that 
> include fixes for those.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Re: Review Request 68440: Added '/tasks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68440/
---

(Updated Aug. 28, 2018, 10:28 a.m.)


Review request for mesos and Alexander Rukletsov.


Changes
---

Rebased onto latest master.


Repository: mesos


Description
---

Added '/tasks' to the set of batched master endpoints.


Diffs (updated)
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68440/diff/2/

Changes: https://reviews.apache.org/r/68440/diff/1-2/


Testing
---

See https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68321: Added '/state-summary' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68321/
---

(Updated Aug. 28, 2018, 10:27 a.m.)


Review request for mesos, Alexander Rukletsov and Benjamin Mahler.


Changes
---

Rebased onto latest master.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/state-summary' to the set of batched master endpoints.


Diffs (updated)
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp 1e01dab8b5160e0b69670c896cb5e9fc0bf31005 


Diff: https://reviews.apache.org/r/68321/diff/4/

Changes: https://reviews.apache.org/r/68321/diff/3-4/


Testing
---

See https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68495: Made command check always waits before removing the nested container.

2018-08-28 Thread Qian Zhang


> On Aug. 28, 2018, 1:16 a.m., Alexander Rukletsov wrote:
> > src/checks/checker_process.cpp
> > Lines 878-889 (original), 890-901 (patched)
> > 
> >
> > It looks like we should always call `waitNestedContainer()` after we 
> > said `previousCheckContainerId = checkContainerId;`. For example here.
> > 
> > Maybe it makes sense to call `waitNestedContainer()` right in the 
> > beginning? We can end up calling it twice, but I think it's fine?

> Maybe it makes sense to call waitNestedContainer() right in the beginning? We 
> can end up calling it twice, but I think it's fine?

That means we will call agent API `WAIT_NESTED_CONTAINER` twice for each 
successful launch of check container, I think that might be a burden for agent 
in a large scale env. So I'd still prefer to call it only in the places where 
we have to do it.


- Qian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68495/#review207984
---


On Aug. 24, 2018, 5:54 p.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68495/
> ---
> 
> (Updated Aug. 24, 2018, 5:54 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Alexander Rukletsov, Gastón Kleiman, 
> and Gilbert Song.
> 
> 
> Bugs: MESOS-8568
> https://issues.apache.org/jira/browse/MESOS-8568
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made command check always waits before removing the nested container.
> 
> 
> Diffs
> -
> 
>   src/checks/checker_process.cpp 77a76f465fe57eab89f027b5acb74c2339551678 
> 
> 
> Diff: https://reviews.apache.org/r/68495/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>



Re: Review Request 68440: Added '/tasks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68440/
---

(Updated Aug. 28, 2018, 10:09 a.m.)


Review request for mesos and Alexander Rukletsov.


Repository: mesos


Description
---

Added '/tasks' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 


Diff: https://reviews.apache.org/r/68440/diff/1/


Testing (updated)
---

https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68440: Added '/tasks' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Line 4171 (original), 4149 (patched)
> > 
> >
> > `return std::move(OK(...));`

Dropping this since we agreed in offline discussions that it would not make a 
difference.


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/master.hpp
> > Lines 1508-1509 (original), 1513-1514 (patched)
> > 
> >
> > // NOTE: Requests to this endpoint are batched.

Dropping this since we agreed in offline discussions to add this comment in a 
separate review.


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68440/#review207954
---


On Aug. 28, 2018, 10:09 a.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68440/
> ---
> 
> (Updated Aug. 28, 2018, 10:09 a.m.)
> 
> 
> Review request for mesos and Alexander Rukletsov.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added '/tasks' to the set of batched master endpoints.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 
> 
> 
> Diff: https://reviews.apache.org/r/68440/diff/1/
> 
> 
> Testing
> ---
> 
> See https://reviews.apache.org/r/68473/
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68321: Added '/state-summary' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68321/
---

(Updated Aug. 28, 2018, 10:06 a.m.)


Review request for mesos, Alexander Rukletsov and Benjamin Mahler.


Bugs: MESOS-9158
https://issues.apache.org/jira/browse/MESOS-9158


Repository: mesos


Description
---

Added '/state-summary' to the set of batched master endpoints.


Diffs
-

  src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
  src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 


Diff: https://reviews.apache.org/r/68321/diff/3/


Testing (updated)
---

https://reviews.apache.org/r/68473/


Thanks,

Benno Evers



Re: Review Request 68321: Added '/state-summary' to the set of batched master endpoints.

2018-08-28 Thread Benno Evers


> On Aug. 27, 2018, 11:19 a.m., Alexander Rukletsov wrote:
> > src/master/http.cpp
> > Line 3476 (original), 3487 (patched)
> > 
> >
> > `return std::move(OK(...));`

Dropping this since we agreed in offline discussions that it would not make a 
difference.


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68321/#review207956
---


On Aug. 21, 2018, 2:09 p.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68321/
> ---
> 
> (Updated Aug. 21, 2018, 2:09 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Benjamin Mahler.
> 
> 
> Bugs: MESOS-9158
> https://issues.apache.org/jira/browse/MESOS-9158
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added '/state-summary' to the set of batched master endpoints.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp ae28d525e5c0fe067ae77a4c71e8f18f772ff1dc 
>   src/master/master.hpp dc0080b24f19b77a4de34ab24aece657726343b8 
> 
> 
> Diff: https://reviews.apache.org/r/68321/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benno Evers
> 
>



Re: Review Request 68535: Updated Python dependencies for 3.7.

2018-08-28 Thread Mesos Reviewbot Windows

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68535/#review208018
---



PASS: Mesos patch 68535 was successfully built and tested.

Reviews applied: `['68535']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2247/mesos-review-68535

- Mesos Reviewbot Windows


On Aug. 28, 2018, 8:05 a.m., Armand Grillet wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68535/
> ---
> 
> (Updated Aug. 28, 2018, 8:05 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Robin Gögge.
> 
> 
> Bugs: MESOS-9186
> https://issues.apache.org/jira/browse/MESOS-9186
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> All the dependencies are now fixed so that we will only need to update
> when a new Python version is being released.
> 
> 
> Diffs
> -
> 
>   src/python/cli_new/pip-requirements.txt 
> 8645912a63b22878f477e96bfa8bc95abe893f85 
>   src/python/cli_new/tox.ini a9a2e69156484a4d172a2746f8cc51089445ae17 
>   src/python/lib/tox.ini 0779fb22d1e8af2dbd17d0fb6b1c8cf3dc2e37d3 
>   support/pip-requirements.txt 89c2de287bc26bbde3006b587a4a8c1bae6f600b 
> 
> 
> Diff: https://reviews.apache.org/r/68535/diff/1/
> 
> 
> Testing
> ---
> 
> Used the scripts and saw that the error describe in MESOS-9186 is gone. 
> Multiple packages had issues with Python 3.7 and I have used versions that 
> include fixes for those.
> 
> 
> Thanks,
> 
> Armand Grillet
> 
>



Review Request 68535: Updated Python dependencies for 3.7.

2018-08-28 Thread Armand Grillet

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68535/
---

Review request for mesos, Benjamin Bannier and Robin Gögge.


Bugs: MESOS-9186
https://issues.apache.org/jira/browse/MESOS-9186


Repository: mesos


Description
---

All the dependencies are now fixed so that we will only need to update
when a new Python version is being released.


Diffs
-

  src/python/cli_new/pip-requirements.txt 
8645912a63b22878f477e96bfa8bc95abe893f85 
  src/python/cli_new/tox.ini a9a2e69156484a4d172a2746f8cc51089445ae17 
  src/python/lib/tox.ini 0779fb22d1e8af2dbd17d0fb6b1c8cf3dc2e37d3 
  support/pip-requirements.txt 89c2de287bc26bbde3006b587a4a8c1bae6f600b 


Diff: https://reviews.apache.org/r/68535/diff/1/


Testing
---

Used the scripts and saw that the error describe in MESOS-9186 is gone. 
Multiple packages had issues with Python 3.7 and I have used versions that 
include fixes for those.


Thanks,

Armand Grillet