Re: Review Request 71436: Replaced removeOffer + recoverResources pairs with specialized helpers.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71436/#review217623
---


Ship it!




Ship It!

- Meng Zhu


On Sept. 6, 2019, 9:56 a.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71436/
> ---
> 
> (Updated Sept. 6, 2019, 9:56 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-1452 and MESOS-9949
> https://issues.apache.org/jira/browse/MESOS-1452
> https://issues.apache.org/jira/browse/MESOS-9949
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds helper methods `Master::rescindOffer()` /
> `Master::discardOffer()` that recover offer's resources in the allocator
> and remove the offer, and replaces paired calls of `removeOffer()` +
> `allocator->recoverResources()` with these helpers.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp 0987d933c03b059c70f0b7c6231df73b8b4d228f 
>   src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
>   src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
>   src/master/quota_handler.cpp f28eb27964afd2c8bfa0bf7a3ab8246442812f2a 
>   src/master/weights_handler.cpp dfb6f06449f85e7acb156cac0ff79cadcc712255 
> 
> 
> Diff: https://reviews.apache.org/r/71436/diff/2/
> 
> 
> Testing
> ---
> 
> make check, 10 times
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71434: Moved `removeOffers()` from `Master::accept()` into `Master::_accept()`.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71434/#review217620
---


Ship it!




Ship It!

- Meng Zhu


On Sept. 6, 2019, 9:55 a.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71434/
> ---
> 
> (Updated Sept. 6, 2019, 9:55 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-1452 and MESOS-9949
> https://issues.apache.org/jira/browse/MESOS-1452
> https://issues.apache.org/jira/browse/MESOS-9949
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch moves offer removal on accept into the deferred continuation
> that follows authorization (if offers pass validation in `accept()`).
> 
> Incrementing the `offers_accepted` metric is also moved to `_accept()`.
> 
> This is a prerequisite for implementing `rescindOffer()` /
> `declineOffer()` / in the dependent patch.
> 
> 
> Diffs
> -
> 
>   src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
>   src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
> 
> 
> Diff: https://reviews.apache.org/r/71434/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71433: Separated handling offer validation failure from handling success.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71433/#review217619
---


Ship it!




Ship It!

- Meng Zhu


On Sept. 6, 2019, 9:50 a.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71433/
> ---
> 
> (Updated Sept. 6, 2019, 9:50 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-1452 and MESOS-9949
> https://issues.apache.org/jira/browse/MESOS-1452
> https://issues.apache.org/jira/browse/MESOS-9949
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch refactors the loop through offer IDs in `Master::accept()`
> into two simpler loops: one loop for the offer validation failure case,
> another for the case of validation success, thus bringing removal of
> offers and recovering their resources close together.
> 
> This is a prerequisite for implementing `rescindOffer()`/
> `declineOffer()` in the dependent patch.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
> 
> 
> Diff: https://reviews.apache.org/r/71433/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71436: Replaced removeOffer + recoverResources pairs with specialized helpers.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71436/#review217617
---



Patch looks great!

Reviews applied: [71433, 71434, 71435, 71436]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Sept. 6, 2019, 4:56 p.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71436/
> ---
> 
> (Updated Sept. 6, 2019, 4:56 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-1452 and MESOS-9949
> https://issues.apache.org/jira/browse/MESOS-1452
> https://issues.apache.org/jira/browse/MESOS-9949
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds helper methods `Master::rescindOffer()` /
> `Master::discardOffer()` that recover offer's resources in the allocator
> and remove the offer, and replaces paired calls of `removeOffer()` +
> `allocator->recoverResources()` with these helpers.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp 0987d933c03b059c70f0b7c6231df73b8b4d228f 
>   src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
>   src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
>   src/master/quota_handler.cpp f28eb27964afd2c8bfa0bf7a3ab8246442812f2a 
>   src/master/weights_handler.cpp dfb6f06449f85e7acb156cac0ff79cadcc712255 
> 
> 
> Diff: https://reviews.apache.org/r/71436/diff/2/
> 
> 
> Testing
> ---
> 
> make check, 10 times
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71428: Removed `Sorter::allocation(const SlaveID& slaveId)`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71428/#review217616
---


Ship it!




Ship It!

- Andrei Sekretenko


On Sept. 3, 2019, 8:50 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71428/
> ---
> 
> (Updated Sept. 3, 2019, 8:50 p.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This paves the way for removing the allocation info tracking
> in the sorter (in favor of tracking them in the allocator).
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 187de173ee2f563e2ff0f7e5c96695dd94a73970 
>   src/master/allocator/mesos/sorter/drf/sorter.hpp 
> f157ec6153325b0457ab1bc76f9eb018712d753a 
>   src/master/allocator/mesos/sorter/drf/sorter.cpp 
> 09889cdf3dc8c0e773b8e2e24154fb0edd2cc254 
>   src/master/allocator/mesos/sorter/random/sorter.hpp 
> 8663ccd4ccdd36c8ea7513b492d3f46a0b15ff5c 
>   src/master/allocator/mesos/sorter/random/sorter.cpp 
> 60a5797472460a8d3d9be938af9f6711ea51d484 
>   src/master/allocator/mesos/sorter/sorter.hpp 
> 52b8a7b57bf17759311b32aa56c26e614119b773 
>   src/tests/sorter_tests.cpp 97ab910cac6e0323012bf229ec32669ea374546a 
> 
> 
> Diff: https://reviews.apache.org/r/71428/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 68508: Added tracking of framework allocations in the allocator Slave class.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68508/#review217615
---


Ship it!




Ship It!

- Andrei Sekretenko


On Sept. 3, 2019, 8:49 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68508/
> ---
> 
> (Updated Sept. 3, 2019, 8:49 p.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This would simplify the tracking logic regarding
> resource allocations in the allocator. See MESOS-9182.
> 
> Negligible performance impact:
> 
> Master:
> 
> BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
> Added 3000 agents in 77.999483ms
> Added 3000 frameworks in 16.736076171secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
> with drf sorter
> Made 3500 allocations in 15.342376944secs
> Made 0 allocation in 13.96720191secs
> 
> Master + this patch:
> 
> BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
> Added 3000 agents in 83.597048ms
> Added 3000 frameworks in 16.757011745secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
> with drf sorter
> Made 3500 allocations in 15.566366241secs
> Made 0 allocation in 14.022591871secs
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> 48ba399e12129b3348b5c43b793f3428acf26d4e 
>   src/master/allocator/mesos/hierarchical.cpp 
> 187de173ee2f563e2ff0f7e5c96695dd94a73970 
> 
> 
> Diff: https://reviews.apache.org/r/68508/diff/3/
> 
> 
> Testing
> ---
> 
> make check
> Benchmark result in the description
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 71436: Replaced removeOffer + recoverResources pairs with specialized helpers.

[Andrei Sekretenko]

> On Sept. 4, 2019, 11:53 p.m., Meng Zhu wrote:
> > src/master/master.cpp
> > Line 3357 (original), 3352 (patched)
> > 
> >
> > Looking at the call site, this `rescind` parameter is redundant, we 
> > could just look at framework->active(), this saves one parameter and also 
> > saves me from wondering what makes a `resind` true.

Looks like we can't :(

rescind = true:
https://github.com/apache/mesos/blob/6c2a94ca0eca90e6d3517e4400f4529ddce0b84c/src/master/master.cpp#L3335
rescind = false:
https://github.com/apache/mesos/blob/6c2a94ca0eca90e6d3517e4400f4529ddce0b84c/src/master/master.cpp#L11228

Both are calling `deactivate()` for an active framework only.


> On Sept. 4, 2019, 11:53 p.m., Meng Zhu wrote:
> > src/master/master.cpp
> > Line 8849 (original), 8811 (patched)
> > 
> >
> > LOG should be "Rescinding"?

yeah, and several other similar places


> On Sept. 4, 2019, 11:53 p.m., Meng Zhu wrote:
> > src/master/master.cpp
> > Line 8910 (original), 8860 (patched)
> > 
> >
> > s/Remove and rescind/Rescind/

Thanks for noticing!

Realized that we have many more comments of this kind. 
Rewrote some of them, removed many others (which are redundant now).


- Andrei


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71436/#review217568
---


On Sept. 6, 2019, 4:56 p.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71436/
> ---
> 
> (Updated Sept. 6, 2019, 4:56 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Meng Zhu.
> 
> 
> Bugs: MESOS-1452 and MESOS-9949
> https://issues.apache.org/jira/browse/MESOS-1452
> https://issues.apache.org/jira/browse/MESOS-9949
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds helper methods `Master::rescindOffer()` /
> `Master::discardOffer()` that recover offer's resources in the allocator
> and remove the offer, and replaces paired calls of `removeOffer()` +
> `allocator->recoverResources()` with these helpers.
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp 0987d933c03b059c70f0b7c6231df73b8b4d228f 
>   src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
>   src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
>   src/master/quota_handler.cpp f28eb27964afd2c8bfa0bf7a3ab8246442812f2a 
>   src/master/weights_handler.cpp dfb6f06449f85e7acb156cac0ff79cadcc712255 
> 
> 
> Diff: https://reviews.apache.org/r/71436/diff/2/
> 
> 
> Testing
> ---
> 
> make check, 10 times
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71436: Replaced removeOffer + recoverResources pairs with specialized helpers.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71436/
---

(Updated Sept. 6, 2019, 4:56 p.m.)


Review request for mesos, Benjamin Mahler and Meng Zhu.


Bugs: MESOS-1452 and MESOS-9949
https://issues.apache.org/jira/browse/MESOS-1452
https://issues.apache.org/jira/browse/MESOS-9949


Repository: mesos


Description
---

This patch adds helper methods `Master::rescindOffer()` /
`Master::discardOffer()` that recover offer's resources in the allocator
and remove the offer, and replaces paired calls of `removeOffer()` +
`allocator->recoverResources()` with these helpers.


Diffs (updated)
-

  src/master/http.cpp 0987d933c03b059c70f0b7c6231df73b8b4d228f 
  src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
  src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 
  src/master/quota_handler.cpp f28eb27964afd2c8bfa0bf7a3ab8246442812f2a 
  src/master/weights_handler.cpp dfb6f06449f85e7acb156cac0ff79cadcc712255 


Diff: https://reviews.apache.org/r/71436/diff/2/

Changes: https://reviews.apache.org/r/71436/diff/1-2/


Testing
---

make check, 10 times


Thanks,

Andrei Sekretenko

Re: Review Request 71434: Moved `removeOffers()` from `Master::accept()` into `Master::_accept()`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71434/
---

(Updated Sept. 6, 2019, 4:55 p.m.)


Review request for mesos, Benjamin Mahler and Meng Zhu.


Bugs: MESOS-1452 and MESOS-9949
https://issues.apache.org/jira/browse/MESOS-1452
https://issues.apache.org/jira/browse/MESOS-9949


Repository: mesos


Description (updated)
---

This patch moves offer removal on accept into the deferred continuation
that follows authorization (if offers pass validation in `accept()`).

Incrementing the `offers_accepted` metric is also moved to `_accept()`.

This is a prerequisite for implementing `rescindOffer()` /
`declineOffer()` / in the dependent patch.


Diffs (updated)
-

  src/master/master.hpp 19c17821d2b8111ebe8e1429a5b4a8423357fdd6 
  src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 


Diff: https://reviews.apache.org/r/71434/diff/2/

Changes: https://reviews.apache.org/r/71434/diff/1-2/


Testing
---


Thanks,

Andrei Sekretenko

Re: Review Request 71433: Separated handling offer validation failure from handling success.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71433/
---

(Updated Sept. 6, 2019, 4:50 p.m.)


Review request for mesos, Benjamin Mahler and Meng Zhu.


Bugs: MESOS-1452 and MESOS-9949
https://issues.apache.org/jira/browse/MESOS-1452
https://issues.apache.org/jira/browse/MESOS-9949


Repository: mesos


Description (updated)
---

This patch refactors the loop through offer IDs in `Master::accept()`
into two simpler loops: one loop for the offer validation failure case,
another for the case of validation success, thus bringing removal of
offers and recovering their resources close together.

This is a prerequisite for implementing `rescindOffer()`/
`declineOffer()` in the dependent patch.


Diffs (updated)
-

  src/master/master.cpp f00906ef2d33920f23127a74ed141fff9d32865b 


Diff: https://reviews.apache.org/r/71433/diff/2/

Changes: https://reviews.apache.org/r/71433/diff/1-2/


Testing
---


Thanks,

Andrei Sekretenko

Re: Review Request 71445: Removed race in `StorageLocalResourceProviderTest.Update`.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71445/#review217612
---



Patch looks great!

Reviews applied: [71445]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Sept. 6, 2019, 12:39 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71445/
> ---
> 
> (Updated Sept. 6, 2019, 12:39 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The agent might take some time before registering with the master which
> could have lead to unexpected retries in this test which the clock was
> resumed. This patch simplifies the test so we do not rely on
> `UpdateSlaveMessage` anymore.
> 
> 
> Diffs
> -
> 
>   src/tests/storage_local_resource_provider_tests.cpp 
> 089aa9787a66d737267179ad461be0c0a99d5c63 
> 
> 
> Diff: https://reviews.apache.org/r/71445/diff/1/
> 
> 
> Testing
> ---
> 
> * `make check`
> * ran test fixed here in repetition under external stress. Previous failure 
> rate ~6%, no failures in >5k runs after
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 71440: Improved allocator inverse offer test.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71440/#review217608
---


Fix it, then Ship it!




I'd suggest being more specific in the description:
s/will not get inverse offers/will not get inverse offers for that agent


src/tests/hierarchical_allocator_tests.cpp
Line 1095 (original), 1127 (patched)


We don't call `updateUnavalability` for agent2 (the slave for which we 
declined an offer) - is this test really able to catch broken 
`Slave::increaseAvailable()`?...


- Andrei Sekretenko


On Sept. 5, 2019, 9:12 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71440/
> ---
> 
> (Updated Sept. 5, 2019, 9:12 p.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The test is augmented to also check that a framework that
> declined offers from an agent will not get inverse offers.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 2c1d0fe2e3ac08e6443cf8348bb72ac7dae454a8 
> 
> 
> Diff: https://reviews.apache.org/r/71440/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 71241: Added a `distcheck` target to the cmake build.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71241/#review217607
---



Bad patch!

Reviews applied: [71240, 71241]

Failed command: ['bash', '-c', "set -o pipefail; export OS='ubuntu:14.04' 
BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose 
--disable-libtool-wrappers --disable-parallel-test-execution' 
ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh 2>&1 | tee 
build_71241"]

Error:
..
ramework aa177ea5-65f5-46eb-9a91-32a999ab669d- (default) at 
scheduler-c294e96f-a4dd-489c-be9d-e20c68060326@172.17.0.3:42379
I0906 12:42:00.742789 18628 sched.cpp:934] Scheduler::resourceOffers took 
114067ns
I0906 12:42:00.748823 18626 http.cpp:1115] HTTP POST for 
/slave(1246)/api/v1/resource_provider from 172.17.0.3:33882
I0906 12:42:00.749876 18607 slave.cpp:8481] Handling resource provider message 
'UPDATE_OPERATION_STATUS: (uuid: e237324c-6556-4c07-b596-ceea66137de5) for 
framework  (latest state: OPERATION_FINISHED, status update state: 
OPERATION_FINISHED)'
I0906 12:42:00.750048 18607 slave.cpp:8934] Updating the state of operation 
with no ID (uuid: e237324c-6556-4c07-b596-ceea66137de5) for an operation API 
call (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I0906 12:42:00.750093 18607 slave.cpp:8688] Forwarding status update of 
operation with no ID (operation_uuid: e237324c-6556-4c07-b596-ceea66137de5) for 
an operator API call
I0906 12:42:00.750433 18619 master.cpp:12271] Updating the state of operation 
'' (uuid: e237324c-6556-4c07-b596-ceea66137de5) for an operator API call 
(latest state: OPERATION_PENDING, status update state: OPERATION_FINISHED)
I0906 12:42:00.751109 18607 slave.cpp:4352] Ignoring new checkpointed resources 
and operations identical to the current version
I0906 12:42:00.826470 18622 status_update_manager_process.hpp:152] Received 
operation status update OPERATION_FINISHED (Status UUID: 
0a86130b-e0a7-4709-8e5a-6f8f95c51e5e) for operation UUID 
478d2252-ba1a-4da2-a023-4de0d3db7b1e on agent 
aa177ea5-65f5-46eb-9a91-32a999ab669d-S0
I0906 12:42:00.826575 18622 status_update_manager_process.hpp:414] Creating 
operation status update stream 478d2252-ba1a-4da2-a023-4de0d3db7b1e 
checkpoint=true
I0906 12:42:00.826617 18615 provider.cpp:498] Received 
ACKNOWLEDGE_OPERATION_STATUS event
I0906 12:42:00.827093 18622 status_update_manager_process.hpp:929] 
Checkpointing UPDATE for operation status update OPERATION_FINISHED (Status 
UUID: 0a86130b-e0a7-4709-8e5a-6f8f95c51e5e) for operation UUID 
478d2252-ba1a-4da2-a023-4de0d3db7b1e on agent 
aa177ea5-65f5-46eb-9a91-32a999ab669d-S0
I0906 12:42:00.909956 18622 status_update_manager_process.hpp:528] Forwarding 
operation status update OPERATION_FINISHED (Status UUID: 
0a86130b-e0a7-4709-8e5a-6f8f95c51e5e) for operation UUID 
478d2252-ba1a-4da2-a023-4de0d3db7b1e on agent 
aa177ea5-65f5-46eb-9a91-32a999ab669d-S0
I0906 12:42:00.910375 18622 status_update_manager_process.hpp:252] Received 
operation status update acknowledgement (UUID: 
5bc541ef-92e0-4b05-ba4e-e4cfb499a7d5) for stream 
e237324c-6556-4c07-b596-ceea66137de5
I0906 12:42:00.910452 18622 status_update_manager_process.hpp:929] 
Checkpointing ACK for operation status update OPERATION_FINISHED (Status UUID: 
5bc541ef-92e0-4b05-ba4e-e4cfb499a7d5) for operation UUID 
e237324c-6556-4c07-b596-ceea66137de5 on agent 
aa177ea5-65f5-46eb-9a91-32a999ab669d-S0
I0906 12:42:00.911002 18612 http_connection.hpp:131] Sending 
UPDATE_OPERATION_STATUS call to 
http://172.17.0.3:42379/slave(1246)/api/v1/resource_provider
I0906 12:42:00.912227 18627 process.cpp:3671] Handling HTTP event for process 
'slave(1246)' with path: '/slave(1246)/api/v1/resource_provider'
I0906 12:42:00.952363 18625 http.cpp:1115] HTTP POST for 
/slave(1246)/api/v1/resource_provider from 172.17.0.3:33882
I0906 12:42:00.953371 18621 slave.cpp:8481] Handling resource provider message 
'UPDATE_OPERATION_STATUS: (uuid: 478d2252-ba1a-4da2-a023-4de0d3db7b1e) for 
framework  (latest state: OPERATION_FINISHED, status update state: 
OPERATION_FINISHED)'
I0906 12:42:00.953565 18621 slave.cpp:8934] Updating the state of operation 
with no ID (uuid: 478d2252-ba1a-4da2-a023-4de0d3db7b1e) for an operation API 
call (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I0906 12:42:00.953629 18621 slave.cpp:8688] Forwarding status update of 
operation with no ID (operation_uuid: 478d2252-ba1a-4da2-a023-4de0d3db7b1e) for 
an operator API call
I0906 12:42:00.954083 18607 master.cpp:12271] Updating the state of operation 
'' (uuid: 478d2252-ba1a-4da2-a023-4de0d3db7b1e) for an operator API call 
(latest state: OPERATION_PENDING, status update state: OPERATION_FINISHED)
I0906 12:42:00.954618 18620 slave.cpp:4352] Ignoring new checkpointed resources 
and operations identical to the current version
I0906 12:42:00.955955 

Review Request 71445: Removed race in `StorageLocalResourceProviderTest.Update`.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71445/
---

Review request for mesos, Chun-Hung Hsiao and Jan Schlicht.


Repository: mesos


Description
---

The agent might take some time before registering with the master which
could have lead to unexpected retries in this test which the clock was
resumed. This patch simplifies the test so we do not rely on
`UpdateSlaveMessage` anymore.


Diffs
-

  src/tests/storage_local_resource_provider_tests.cpp 
089aa9787a66d737267179ad461be0c0a99d5c63 


Diff: https://reviews.apache.org/r/71445/diff/1/


Testing
---

* `make check`
* ran test fixed here in repetition under external stress. Previous failure 
rate ~6%, no failures in >5k runs after


Thanks,

Benjamin Bannier

Re: Review Request 71240: Added a `dist` target to the cmake build.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71240/
---

(Updated Sept. 6, 2019, 12:13 p.m.)


Review request for mesos, Benjamin Mahler and Till Toenshoff.


Repository: mesos


Description
---

This patch adds a `dist` target to the cmake build, analogous to the
target provided by the autotools build.

While cmake already provides a `package_source` target to create a
source archive it does not take care of only including relevant files,
but instead adds all files to the archive, e.g., including build or
backup files which should not be part of a release. For that reason this
patch introduces a script which performs a temporary `git clone` from
the checked out git repository and creates the archive from that clean
tree.

This patch also removes a hardcoded list of ignored files which was
by design not exhaustive.


Diffs (updated)
-

  CMakeLists.txt dc50dd457585c14a456ca710da4a16a0d45ef17a 
  cmake/dist.sh PRE-CREATION 


Diff: https://reviews.apache.org/r/71240/diff/3/

Changes: https://reviews.apache.org/r/71240/diff/2-3/


Testing
---

`ninja dist` produces a tarball, no artifacts left around but tarball.


Thanks,

Benjamin Bannier

Re: Review Request 71441: Fixed Docker registry malformed URI contruction.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71441/#review217605
---



Bad patch!

Reviews applied: [71441]

Failed command: ['bash', '-c', "set -o pipefail; export OS='ubuntu:14.04' 
BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose 
--disable-libtool-wrappers --disable-parallel-test-execution' 
ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh 2>&1 | tee 
build_71441"]

Error:
..
9:24:09.606482 18639 hierarchical.cpp:1726] Performed allocation for 1 agents 
in 987973ns
I0906 09:24:09.606830 18639 hierarchical.cpp:1726] Performed allocation for 1 
agents in 182810ns
I0906 09:24:09.607092 18638 master.cpp:10432] Sending offers [ 
7fc159b6-c052-42d9-b381-854540f8c3e1-O4 ] to framework 
7fc159b6-c052-42d9-b381-854540f8c3e1- (default) at 
scheduler-3cd6d54a-616a-4497-a48b-77109beb52c6@172.17.0.3:36221
I0906 09:24:09.607820 18627 sched.cpp:934] Scheduler::resourceOffers took 
85724ns
I0906 09:24:09.616456 18635 http.cpp:1115] HTTP POST for 
/slave(1246)/api/v1/resource_provider from 172.17.0.3:58998
I0906 09:24:09.617684 18631 slave.cpp:8481] Handling resource provider message 
'UPDATE_OPERATION_STATUS: (uuid: 5eb6008c-dc46-4734-84e3-ccf1735ee327) for 
framework  (latest state: OPERATION_FINISHED, status update state: 
OPERATION_FINISHED)'
I0906 09:24:09.617880 18631 slave.cpp:8934] Updating the state of operation 
with no ID (uuid: 5eb6008c-dc46-4734-84e3-ccf1735ee327) for an operation API 
call (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I0906 09:24:09.617925 18631 slave.cpp:8688] Forwarding status update of 
operation with no ID (operation_uuid: 5eb6008c-dc46-4734-84e3-ccf1735ee327) for 
an operator API call
I0906 09:24:09.618274 18622 master.cpp:12271] Updating the state of operation 
'' (uuid: 5eb6008c-dc46-4734-84e3-ccf1735ee327) for an operator API call 
(latest state: OPERATION_PENDING, status update state: OPERATION_FINISHED)
I0906 09:24:09.618831 18634 slave.cpp:4352] Ignoring new checkpointed resources 
and operations identical to the current version
I0906 09:24:09.718955 18627 status_update_manager_process.hpp:152] Received 
operation status update OPERATION_FINISHED (Status UUID: 
3921e652-54f4-4e59-b058-a26fc5622ace) for operation UUID 
13b0617f-06ad-427b-8d90-803d7271dfc9 on agent 
7fc159b6-c052-42d9-b381-854540f8c3e1-S0
I0906 09:24:09.719056 18627 status_update_manager_process.hpp:414] Creating 
operation status update stream 13b0617f-06ad-427b-8d90-803d7271dfc9 
checkpoint=true
I0906 09:24:09.719115 18637 provider.cpp:498] Received 
ACKNOWLEDGE_OPERATION_STATUS event
I0906 09:24:09.719399 18627 status_update_manager_process.hpp:929] 
Checkpointing UPDATE for operation status update OPERATION_FINISHED (Status 
UUID: 3921e652-54f4-4e59-b058-a26fc5622ace) for operation UUID 
13b0617f-06ad-427b-8d90-803d7271dfc9 on agent 
7fc159b6-c052-42d9-b381-854540f8c3e1-S0
I0906 09:24:09.810786 18627 status_update_manager_process.hpp:528] Forwarding 
operation status update OPERATION_FINISHED (Status UUID: 
3921e652-54f4-4e59-b058-a26fc5622ace) for operation UUID 
13b0617f-06ad-427b-8d90-803d7271dfc9 on agent 
7fc159b6-c052-42d9-b381-854540f8c3e1-S0
I0906 09:24:09.811287 18627 status_update_manager_process.hpp:252] Received 
operation status update acknowledgement (UUID: 
ee7d1ff5-502a-49b4-91e9-a532d5279ee0) for stream 
5eb6008c-dc46-4734-84e3-ccf1735ee327
I0906 09:24:09.811357 18627 status_update_manager_process.hpp:929] 
Checkpointing ACK for operation status update OPERATION_FINISHED (Status UUID: 
ee7d1ff5-502a-49b4-91e9-a532d5279ee0) for operation UUID 
5eb6008c-dc46-4734-84e3-ccf1735ee327 on agent 
7fc159b6-c052-42d9-b381-854540f8c3e1-S0
I0906 09:24:09.811977 18626 http_connection.hpp:131] Sending 
UPDATE_OPERATION_STATUS call to 
http://172.17.0.3:36221/slave(1246)/api/v1/resource_provider
I0906 09:24:09.813196 18628 process.cpp:3671] Handling HTTP event for process 
'slave(1246)' with path: '/slave(1246)/api/v1/resource_provider'
I0906 09:24:09.852684 18634 http.cpp:1115] HTTP POST for 
/slave(1246)/api/v1/resource_provider from 172.17.0.3:58998
I0906 09:24:09.853847 18625 slave.cpp:8481] Handling resource provider message 
'UPDATE_OPERATION_STATUS: (uuid: 13b0617f-06ad-427b-8d90-803d7271dfc9) for 
framework  (latest state: OPERATION_FINISHED, status update state: 
OPERATION_FINISHED)'
I0906 09:24:09.854044 18625 slave.cpp:8934] Updating the state of operation 
with no ID (uuid: 13b0617f-06ad-427b-8d90-803d7271dfc9) for an operation API 
call (latest state: OPERATION_FINISHED, status update state: OPERATION_FINISHED)
I0906 09:24:09.854100 18625 slave.cpp:8688] Forwarding status update of 
operation with no ID (operation_uuid: 13b0617f-06ad-427b-8d90-803d7271dfc9) for 
an operator API call
I0906 09:24:09.854467 18624 master.cpp:12271] Updating the state of operation 
'' (uuid