Re: Review Request 43999: Use relative path to create libraries symbolic link.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43999/#review121033 --- Ship it! ran a patched version through our pipeline and our rpm builds look good, thanks! - Jacob Janco On Feb. 26, 2016, 8:18 a.m., Zhiwei Chen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/43999/ > --- > > (Updated Feb. 26, 2016, 8:18 a.m.) > > > Review request for mesos, James Peach and Kapil Arya. > > > Bugs: MESOS-4774 > https://issues.apache.org/jira/browse/MESOS-4774 > > > Repository: mesos > > > Description > --- > > Use relative path to create libraries symbolic link. > > > Diffs > - > > src/Makefile.am 2a26261b513bb7c03437ed8e850c3b36b93d82f5 > > Diff: https://reviews.apache.org/r/43999/diff/ > > > Testing > --- > > After make install, the library links will be relative path. > > > Thanks, > > Zhiwei Chen > >
Review Request 46415: construct http reponse body in libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46415/ --- Review request for mesos and Alexander Rukletsov. Bugs: MESOS-4126 https://issues.apache.org/jira/browse/MESOS-4126 Repository: mesos Description --- construct MethodNotAllowed http reponse body in libprocess Diffs - 3rdparty/libprocess/include/process/http.hpp 8f4eabcbb71ead1f5c28e1d8a2dd40db7af1f297 Diff: https://reviews.apache.org/r/46415/diff/ Testing --- Thanks, Jacob Janco
Review Request 46416: remove hardcoding for MethodNotAllowed for http
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46416/ --- Review request for mesos and Alexander Rukletsov. Bugs: MESOS-4126 https://issues.apache.org/jira/browse/MESOS-4126 Repository: mesos Description --- remove hardcoding for MethodNotAllowed for http Diffs - src/master/http.cpp a9cb99a92ff5a783e719df5e5cfb6e8301241df9 src/slave/http.cpp 3908e33ed5b233387790276f6f5d884452087d2c Diff: https://reviews.apache.org/r/46416/diff/ Testing --- Thanks, Jacob Janco
Review Request 46504: Construct error string in MethodNotAllowed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46504/ --- Review request for mesos and Alexander Rukletsov. Bugs: MESOS-4126 https://issues.apache.org/jira/browse/MESOS-4126 Repository: mesos Description --- Construct error string in MethodNotAllowed Diffs - 3rdparty/libprocess/include/process/http.hpp 8f4eabcbb71ead1f5c28e1d8a2dd40db7af1f297 Diff: https://reviews.apache.org/r/46504/diff/ Testing --- Thanks, Jacob Janco
Review Request 46505: Remove MethodNotAllowed error string creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46505/ --- Review request for mesos and Alexander Rukletsov. Bugs: MESOS-4126 https://issues.apache.org/jira/browse/MESOS-4126 Repository: mesos Description --- Remove MethodNotAllowed error string creation Diffs - src/master/http.cpp a9cb99a92ff5a783e719df5e5cfb6e8301241df9 src/slave/http.cpp 3908e33ed5b233387790276f6f5d884452087d2c Diff: https://reviews.apache.org/r/46505/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Oct. 4, 2016, 9:47 p.m., Guangya Liu wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 1306 > > <https://reviews.apache.org/r/51027/diff/9/?file=1522284#file1522284line1306> > > > > The `addFramework` will call `allocate()` and allocate `all agents > > resources` but not a `single agnent`, so here we should not mention > > `addFramework`, but only the following three: > > > > 1) addSlave > > 2) updateSlave > > 3) updateUnavailability? I mentioend addFramework and addSlave intentionally so we have an example of allocating all known slaves as well as for a single slave. In both cases we will take the union of allocationCandidates. For example I addSlave(agent1) and immediately addFramework() ... an allocation would be queued for the single agent then all agents would be added in the union when addFramework() is called. When the actual allocation run occurs all slaves known at the time of addFramework() will be included in allocationCandidates. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review151408 ------- On Oct. 4, 2016, 11:31 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Oct. 4, 2016, 11:31 p.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > 2c31471ee0f5d6836393bf87ff9ecfd8df835013 > src/master/allocator/mesos/hierarchical.cpp > c8f9492ee1b69e125a1e841116d22a578a9b524e > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 and > https://reviews.apache.org/r/52534 > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Oct. 18, 2016, 4:14 p.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 2c31471ee0f5d6836393bf87ff9ecfd8df835013 src/master/allocator/mesos/hierarchical.cpp c8f9492ee1b69e125a1e841116d22a578a9b524e Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 and https://reviews.apache.org/r/52534 Thanks, Jacob Janco
Review Request 53479: Perform GC asynchronously.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/ --- Review request for mesos. Repository: mesos Description --- Perform GC asynchronously. Diffs - src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 Diff: https://reviews.apache.org/r/53479/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 53479: Perform GC asynchronously.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/ --- (Updated Nov. 4, 2016, 5 p.m.) Review request for mesos. Repository: mesos Description --- Perform GC asynchronously. Diffs - src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 Diff: https://reviews.apache.org/r/53479/diff/ Testing (updated) --- `make check` Thanks, Jacob Janco
Re: Review Request 53479: Perform GC asynchronously.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/ --- (Updated Nov. 4, 2016, 5:25 p.m.) Review request for mesos. Bugs: MESOS-6549 https://issues.apache.org/jira/browse/MESOS-6549 Repository: mesos Description --- Perform GC asynchronously. Diffs - src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 Diff: https://reviews.apache.org/r/53479/diff/ Testing --- `make check` Thanks, Jacob Janco
Re: Review Request 53479: Perform GC asynchronously.
4435line188> > > > > I think we actually have to use the following? > > > > ``` > > map> paths; > > ``` > > > > Because we rmdir asynchronously, when it's finished we need to retrieve > > the same `PathInfo` by both the removalTime and the path! > > > > If we do that, we can access all entries by reference so we won't have > > this problem. However we need to be cautious in cleaning up the outer entry > > when the inner map is all cleaned up. Ah hmmm are you saying instead of carrying the timeouts and paths data structures, carry a timeout to path/pathinfo map? I changed the logic to use `paths.get(removalTime)` and index into the new `futures` hashmap with the list of `pathInfo`. Maybe this is cleaner? > On Nov. 12, 2016, 12:48 a.m., Jiang Yan Xu wrote: > > src/slave/gc.cpp, line 216 > > <https://reviews.apache.org/r/53479/diff/1/?file=1554435#file1554435line216> > > > > It's probably safe to pass a copy of `PathInfo` here but the fact that > > it's safe it's not obvious. > > > > We can probably just pass a `path` so in the continuation we can use > > `removalTime` and `path` to get the `PathInfo` back and we can just `CHECK` > > to assert that the entry should be there and add a comment on why it should > > be there (because we don't erase the any entry with a pending `rmdir`, we > > wait until it's finished). Removed in favor of looking up in `_remove` > On Nov. 12, 2016, 12:48 a.m., Jiang Yan Xu wrote: > > src/slave/gc.cpp, lines 126-131 > > <https://reviews.apache.org/r/53479/diff/1/?file=1554435#file1554435line126> > > > > As it's written this way (along with the single path `_remove()`), in > > each batch of `rmdir()`s, the quickest one is going to trigger the next > > reset, by which time the head of the `paths` could be the same batch. > > > > We won't have this problem if we invoke `_remove()` once per "batch", > > in which case we don't need to change this method here. Right, this is unecessary with the batching since there is not trampling on toes when `_remove` streams in for each op. - Jacob ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/#review155639 --- On Nov. 12, 2016, 12:48 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53479/ > --- > > (Updated Nov. 12, 2016, 12:48 a.m.) > > > Review request for mesos, Benjamin Mahler and Jiang Yan Xu. > > > Bugs: MESOS-6549 > https://issues.apache.org/jira/browse/MESOS-6549 > > > Repository: mesos > > > Description > --- > > Perform GC asynchronously. > > > Diffs > - > > src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e > src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 > > Diff: https://reviews.apache.org/r/53479/diff/ > > > Testing > --- > > `make check` > > > Thanks, > > Jacob Janco > >
Re: Review Request 53479: Perform agent GC asynchronously.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/ --- (Updated Dec. 2, 2016, 2:58 a.m.) Review request for mesos, Benjamin Mahler and Jiang Yan Xu. Summary (updated) - Perform agent GC asynchronously. Bugs: MESOS-6549 https://issues.apache.org/jira/browse/MESOS-6549 Repository: mesos Description (updated) --- - Previously, rmdir operations were serialized through the garbage collector. Expensive removal events had the potential to delay task launch. - MESOS-6549 Diffs (updated) - src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 Diff: https://reviews.apache.org/r/53479/diff/ Testing --- `make check` Thanks, Jacob Janco
Re: Review Request 53479: Perform GC asynchronously.
> On Nov. 21, 2016, 6:47 p.m., Benjamin Mahler wrote: > > Thanks for fixing this! It would be great to include some of the context > > from the ticket so that the commit message provides enough context (and/or > > points to the JIRA). In particular, that before this change, the launching > > of other tasks could be delayed during expensive rmdir operations since > > they all have to be serialized through the garbage collector. Will take a > > closer look after Yan's review. Ah ok added more context to the commit message. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/#review156481 --- On Nov. 12, 2016, 12:48 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53479/ > --- > > (Updated Nov. 12, 2016, 12:48 a.m.) > > > Review request for mesos, Benjamin Mahler and Jiang Yan Xu. > > > Bugs: MESOS-6549 > https://issues.apache.org/jira/browse/MESOS-6549 > > > Repository: mesos > > > Description > --- > > Perform GC asynchronously. > > > Diffs > - > > src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e > src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 > > Diff: https://reviews.apache.org/r/53479/diff/ > > > Testing > --- > > `make check` > > > Thanks, > > Jacob Janco > >
Re: Review Request 53479: Perform agent GC asynchronously.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53479/ --- (Updated Dec. 2, 2016, 3:01 a.m.) Review request for mesos, Benjamin Mahler and Jiang Yan Xu. Bugs: MESOS-6549 https://issues.apache.org/jira/browse/MESOS-6549 Repository: mesos Description --- - Previously, rmdir operations were serialized through the garbage collector. Expensive removal events had the potential to delay task launch. - MESOS-6549 Diffs - src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 Diff: https://reviews.apache.org/r/53479/diff/ Testing (updated) --- `./bin/mesos-tests.sh --gtest_filter="GarbageCollector*" --gtest_repeat=100 --gtest_break_on_failure` Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Dec. 12, 2016, 9:30 a.m., Jiang Yan Xu wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1307
> > <https://reviews.apache.org/r/51027/diff/10/?file=1540692#file1540692line1307>
> >
> > s/a single slaveId/a single `slaveId`/ to be consistent.
Changed the wording to describe the events in plain english, previously I was
describing methods.
// Events that trigger allocations, e.g. adding an agent or a framework,
// update the set of `allocationCandidates` with a single agent in the
// former case and the set of all known agents in the latter.
> On Dec. 12, 2016, 9:30 a.m., Jiang Yan Xu wrote:
> > src/master/allocator/mesos/hierarchical.cpp, lines 1287-1288
> > <https://reviews.apache.org/r/51027/diff/10/?file=1540692#file1540692line1287>
> >
> > Does this work?
> >
> > ```
> > return allocate({slaveId});
> > ```
No, the hashset constructor allows an initializer list, I think `return
allocate({slaveId});` would just scope `slaveId`.
- Jacob
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review158818
-------
On Oct. 18, 2016, 4:14 p.m., Jacob Janco wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> ---
>
> (Updated Oct. 18, 2016, 4:14 p.m.)
>
>
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus
> Ma, and Jiang Yan Xu.
>
>
> Bugs: MESOS-3157
> https://issues.apache.org/jira/browse/MESOS-3157
>
>
> Repository: mesos
>
>
> Description
> ---
>
> - Triggered allocations dispatch allocate() only
> if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
> cleared when enqueued allocations are processed.
>
>
> Diffs
> -
>
> src/master/allocator/mesos/hierarchical.hpp
> 2c31471ee0f5d6836393bf87ff9ecfd8df835013
> src/master/allocator/mesos/hierarchical.cpp
> c8f9492ee1b69e125a1e841116d22a578a9b524e
>
> Diff: https://reviews.apache.org/r/51027/diff/
>
>
> Testing
> ---
>
> make check
>
> note: check without filters depends on https://reviews.apache.org/r/51028 and
> https://reviews.apache.org/r/52534
>
>
> Thanks,
>
> Jacob Janco
>
>
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Dec. 20, 2016, 7:20 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 and https://reviews.apache.org/r/52534 Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Dec. 20, 2016, 7:35 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/51027/diff/ Testing (updated) --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Sept. 28, 2016, 11:02 p.m., Guangya Liu wrote: > > src/tests/hierarchical_allocator_tests.cpp, line 2974 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line2974> > > > > Can you please add some comments here for why not using > > `Clock::pause()`? Added this back. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review150773 --- On Sept. 28, 2016, 9:16 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Sept. 28, 2016, 9:16 p.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 46553f6d501deb37862dd5ba48be1c114f6e6cb8 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Dec. 13, 2016, 4:30 p.m., Jiang Yan Xu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3005-3006 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line3005> > > > > Similar to Guangya's suggestion, I think we can elaborate on the reason > > to use tow agents and two frameworks. > > > > ``` > > Trigger at least two allocation runs to generate the window statistics. > > Here we are using two pairs of `addSlave()` and `addFramework()` calls > > because each guarantees one allocation run that yields an `Allocation`. > > ``` I realized all we need is to await the allocation after `addSlave()` as well as `addFramework()` to ensure that we have window metrics. > On Dec. 13, 2016, 4:30 p.m., Jiang Yan Xu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3621-3629 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line3621> > > > > I think we are still intereted in the number of allocations and we > > expect to see them decrease because of batching. We don't use EXPECT on > > them but we print them out as part of the benchmark? Added this in. Ex test output: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 Using 1000 agents and 1000 frameworks Added 1000 frameworks in 18064us Added 1000 agents in 5.229825secs performing 1000 allocations Updated 1000 agents in 5.39987secs performing 1000 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 (10708 ms) - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review158967 --- On Sept. 28, 2016, 9:16 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Sept. 28, 2016, 9:16 p.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 46553f6d501deb37862dd5ba48be1c114f6e6cb8 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Sept. 29, 2016, 3:06 a.m., Guangya Liu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3623-3627 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line3623> > > > > Instead of killing this part, what about updating `offerCallback` > > function to record the number of `agents`? And then assert the number of > > `agents` here is same as number of agents in cluster. Added in logging for the offer callbacks which do correspond to actual runs performed as suggested below, what do you think? - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review150798 ------- On Sept. 28, 2016, 9:16 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Sept. 28, 2016, 9:16 p.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 46553f6d501deb37862dd5ba48be1c114f6e6cb8 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix broken tests post batched event allocation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Dec. 20, 2016, 12:05 p.m.) Review request for mesos and Jiang Yan Xu. Summary (updated) - Fix broken tests post batched event allocation. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51028: Fix broken tests post batched event allocation.
> On Dec. 13, 2016, 4:30 p.m., Jiang Yan Xu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3621-3629 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line3621> > > > > I think we are still intereted in the number of allocations and we > > expect to see them decrease because of batching. We don't use EXPECT on > > them but we print them out as part of the benchmark? > > Jacob Janco wrote: > Added this in. Ex test output: > > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 > Using 1000 agents and 1000 frameworks > Added 1000 frameworks in 18064us > Added 1000 agents in 5.229825secs performing 1000 allocations > Updated 1000 agents in 5.39987secs performing 1000 allocations > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 > (10708 ms) A better snippet showing trend: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0 Using 1000 agents and 1 frameworks Added 1 frameworks in 866us Added 1000 agents in 616149us performing 5 allocations Updated 1000 agents in 351825us performing 3 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0 (1014 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/1 Using 1000 agents and 50 frameworks Added 50 frameworks in 952us Added 1000 agents in 744594us performing 142 allocations Updated 1000 agents in 508415us performing 111 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/1 (1310 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/2 Using 1000 agents and 100 frameworks Added 100 frameworks in 1877us Added 1000 agents in 1.088271secs performing 179 allocations Updated 1000 agents in 867190us performing 135 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/2 (2006 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/3 Using 1000 agents and 200 frameworks Added 200 frameworks in 3357us Added 1000 agents in 1.516704secs performing 294 allocations Updated 1000 agents in 1.319724secs performing 215 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/3 (2894 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/4 Using 1000 agents and 500 frameworks Added 500 frameworks in 8704us Added 1000 agents in 2.890338secs performing 685 allocations Updated 1000 agents in 2.783646secs performing 513 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/4 (5734 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 Using 1000 agents and 1000 frameworks Added 1000 frameworks in 16440us Added 1000 agents in 5.211817secs performing 1000 allocations Updated 1000 agents in 5.439422secs performing 1000 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/5 (10726 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/6 Using 1000 agents and 3000 frameworks Added 3000 frameworks in 46501us Added 1000 agents in 14.495025secs performing 1000 allocations Updated 1000 agents in 14.99748secs performing 1000 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/6 (29611 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/7 Using 1000 agents and 6000 frameworks Added 6000 frameworks in 83773us Added 1000 agents in 28.992552secs performing 1000 allocations Updated 1000 agents in 29.068944secs performing 1000 allocations [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/7 (58237 ms) - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review158967 ------- On Dec. 20, 2016, 12:05 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Dec. 20, 2016, 12:05 p.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Dec. 21, 2016, 5:58 a.m.) Review request for mesos and Jiang Yan Xu. Changes --- Fix AllocationRunTimerMetrics logic. Summary (updated) - Fix tests with rapidly triggered allocations. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Sept. 29, 2016, 3:06 a.m., Guangya Liu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3007-3011 > > <https://reviews.apache.org/r/51028/diff/4/?file=1513695#file1513695line3007> > > > > Just a question here, can we always guarantee only one `allocate()` > > will be called? I think that this depends on the speed of dispatching the > > `run` request. Right, I took a look at this. I believe pausing the clock between the addSlave and addFramework messages is correct, this does depend on speed. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review150798 ------- On Dec. 21, 2016, 5:58 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Dec. 21, 2016, 5:58 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Dec. 21, 2016, 6:02 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 52534: Dispatch filter expiration twice.
> On Dec. 12, 2016, 5:33 p.m., Alexander Rukletsov wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 1084-1085 > > <https://reviews.apache.org/r/52534/diff/1/?file=1522444#file1522444line1084> > > > > If you want to add such a TODO, it should probably live close to > > `recoverResources()`. Removed this TODO. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/#review158879 --- On Dec. 21, 2016, 6:07 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52534/ > --- > > (Updated Dec. 21, 2016, 6:07 a.m.) > > > Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan > Xu. > > > Repository: mesos > > > Description > --- > > - With an asynchronous `batch()` allocation, > this ensures that filters do not expire > before the next allocation. > - This patch should be reverted when allocation > occurs on resource recovery. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > a6424d624864155e1c87a28a63b784512c5c8722 > src/master/allocator/mesos/hierarchical.cpp > 91b1ec43940a788459f045ca4a4b82d4e8373bca > > Diff: https://reviews.apache.org/r/52534/diff/ > > > Testing > --- > > With https://reviews.apache.org/r/51027/: > > GTEST_FILTER="-*SmallOfferFilter*" make check -j8 > > > Thanks, > > Jacob Janco > >
Re: Review Request 52534: Dispatch filter expiration twice.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/ --- (Updated Dec. 21, 2016, 6:07 a.m.) Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan Xu. Repository: mesos Description --- - With an asynchronous `batch()` allocation, this ensures that filters do not expire before the next allocation. - This patch should be reverted when allocation occurs on resource recovery. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/52534/diff/ Testing --- With https://reviews.apache.org/r/51027/: GTEST_FILTER="-*SmallOfferFilter*" make check -j8 Thanks, Jacob Janco
Re: Review Request 52534: Dispatch filter expiration twice.
> On Dec. 12, 2016, 5 p.m., Jiang Yan Xu wrote: > > I feel it's hard to explain this patch without /r/51027/. It's probably > > better to have it go after /r/51027/ in the chain. It's fine if /r/51027/ > > requires this patch to pass the tests. Yep, this this patch is necessary because of 51027. I'll chain accordingly. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/#review158858 ------- On Dec. 21, 2016, 6:07 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52534/ > --- > > (Updated Dec. 21, 2016, 6:07 a.m.) > > > Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan > Xu. > > > Repository: mesos > > > Description > --- > > - With an asynchronous `batch()` allocation, > this ensures that filters do not expire > before the next allocation. > - This patch should be reverted when allocation > occurs on resource recovery. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > a6424d624864155e1c87a28a63b784512c5c8722 > src/master/allocator/mesos/hierarchical.cpp > 91b1ec43940a788459f045ca4a4b82d4e8373bca > > Diff: https://reviews.apache.org/r/52534/diff/ > > > Testing > --- > > With https://reviews.apache.org/r/51027/: > > GTEST_FILTER="-*SmallOfferFilter*" make check -j8 > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Jan. 12, 2017, 6:55 p.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Changes --- Change JIRA. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 52534: Dispatch filter expiration twice.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/ --- (Updated Jan. 12, 2017, 6:56 p.m.) Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan Xu. Changes --- Change JIRA. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - With an asynchronous `batch()` allocation, this ensures that filters do not expire before the next allocation. - This patch should be reverted when allocation occurs on resource recovery. Diffs - src/master/allocator/mesos/hierarchical.hpp a6424d624864155e1c87a28a63b784512c5c8722 src/master/allocator/mesos/hierarchical.cpp 91b1ec43940a788459f045ca4a4b82d4e8373bca Diff: https://reviews.apache.org/r/52534/diff/ Testing --- With https://reviews.apache.org/r/51027/: GTEST_FILTER="-*SmallOfferFilter*" make check -j8 Thanks, Jacob Janco
Review Request 55691: Fix XSS vulnerability in pailer invocation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55691/ --- Review request for mesos. Repository: mesos Description --- Fix XSS vulnerability in pailer invocation. Diffs - src/webui/master/static/js/controllers.js 388ca2447716cbc7141da6a20daf2340621a16e8 src/webui/master/static/pailer.html 19e0981143bd7e8372b49f4f036867e9dd05727a Diff: https://reviews.apache.org/r/55691/diff/ Testing --- make -j8 + test framework + checking pailer representation of files in sandbox Thanks, Jacob Janco
Re: Review Request 55691: Fix XSS vulnerability in pailer invocation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55691/ --- (Updated Jan. 18, 2017, 11:37 p.m.) Review request for mesos and Jiang Yan Xu. Bugs: MESOS-6947 https://issues.apache.org/jira/browse/MESOS-6947 Repository: mesos Description --- Fix XSS vulnerability in pailer invocation. Diffs - src/webui/master/static/js/controllers.js 388ca2447716cbc7141da6a20daf2340621a16e8 src/webui/master/static/pailer.html 19e0981143bd7e8372b49f4f036867e9dd05727a Diff: https://reviews.apache.org/r/55691/diff/ Testing --- make -j8 + test framework + checking pailer representation of files in sandbox Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 18, 2017, 11:29 p.m.) Review request for mesos and Jiang Yan Xu. Changes --- Change JIRA. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 55691: Fix XSS vulnerability in pailer invocation.
> On Jan. 19, 2017, 4:27 p.m., haosdent huang wrote: > > Hi, seems set `document.cookie` could work instead of use localstorage. The > > problem of localstorage is not supported some old browsers. Have you try > > set cookie before? I think the attack vector would be similar if we were to pass the url through a cookie. localStorage is supported by: Feature Chrome Firefox (Gecko) Internet Explorer Opera Safari (WebKit) localStorage4 3.5 8 10.50 4 sessionStorage 5 2 8 10.50 4 I think this is fairly good coverage especially considering Microsoft's end of support for legacy browsers. If it becomes an issue we can definitely rethink this. > On Jan. 19, 2017, 4:27 p.m., haosdent huang wrote: > > src/webui/master/static/pailer.html, lines 46-68 > > <https://reviews.apache.org/r/55691/diff/1/?file=1608498#file1608498line46> > > > > I think we remove this snippet? This block of code keeps localStorage clean and sets the sessionStorage for the life of the open pailer window, so we need to persist the value through reloads. > On Jan. 19, 2017, 4:27 p.m., haosdent huang wrote: > > src/webui/master/static/pailer.html, line 80 > > <https://reviews.apache.org/r/55691/diff/1/?file=1608498#file1608498line80> > > > > I think we could `localStorage.getItem/removeItem` above and use it > > here directly? storageKey is scoped above this to keep it out of the global namespace - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55691/#review162303 --- On Jan. 18, 2017, 11:40 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55691/ > --- > > (Updated Jan. 18, 2017, 11:40 p.m.) > > > Review request for mesos, haosdent huang and Jiang Yan Xu. > > > Bugs: MESOS-6947 > https://issues.apache.org/jira/browse/MESOS-6947 > > > Repository: mesos > > > Description > --- > > Fix XSS vulnerability in pailer invocation. > > > Diffs > - > > src/webui/master/static/js/controllers.js > 388ca2447716cbc7141da6a20daf2340621a16e8 > src/webui/master/static/pailer.html > 19e0981143bd7e8372b49f4f036867e9dd05727a > > Diff: https://reviews.apache.org/r/55691/diff/ > > > Testing > --- > > make -j8 + test framework + checking pailer representation of files in sandbox > > > Thanks, > > Jacob Janco > >
Review Request 55791: Rework clipboard functionality in UI.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55791/ --- Review request for mesos. Repository: mesos Description --- - Use pure js clipboard.js and remove flash dependency. - Vendored zeroclipboard has security issues https://www.cvedetails.com/cve/CVE-2014-1869/ Diffs - src/webui/master/static/css/mesos.css c817aaee0fd2d2579ff2e4a152d47e83ce80d68c src/webui/master/static/index.html a4eda3034906d100a20dcd0a2473112c1edd6b42 src/webui/master/static/js/app.js c32177aa23c962f2bdf03d98272fb6d21a565382 src/webui/master/static/js/clipboard-1.5.16.js PRE-CREATION src/webui/master/static/js/clipboard-1.5.16.min.js PRE-CREATION src/webui/master/static/js/zeroclipboard-1.1.7.js d45551d6c9f75c275032219768549f76865b9cd3 src/webui/master/static/js/zeroclipboard-1.1.7.min.js 32535fddf1baab255fcbf5d3d776a0a9ac1447e8 src/webui/master/static/obj/zeroclipboard-1.1.7.swf 880e64ee7614e224660c6616b4bbb1ee5fab8ec9 Diff: https://reviews.apache.org/r/55791/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 55791: Rework clipboard functionality in UI.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55791/ --- (Updated Jan. 22, 2017, 6:11 p.m.) Review request for mesos and haosdent huang. Bugs: MESOS-6969 https://issues.apache.org/jira/browse/MESOS-6969 Repository: mesos Description --- - Use pure js clipboard.js and remove flash dependency. - Vendored zeroclipboard has security issues https://www.cvedetails.com/cve/CVE-2014-1869/ Diffs - src/webui/master/static/css/mesos.css c817aaee0fd2d2579ff2e4a152d47e83ce80d68c src/webui/master/static/index.html a4eda3034906d100a20dcd0a2473112c1edd6b42 src/webui/master/static/js/app.js c32177aa23c962f2bdf03d98272fb6d21a565382 src/webui/master/static/js/clipboard-1.5.16.js PRE-CREATION src/webui/master/static/js/clipboard-1.5.16.min.js PRE-CREATION src/webui/master/static/js/zeroclipboard-1.1.7.js d45551d6c9f75c275032219768549f76865b9cd3 src/webui/master/static/js/zeroclipboard-1.1.7.min.js 32535fddf1baab255fcbf5d3d776a0a9ac1447e8 src/webui/master/static/obj/zeroclipboard-1.1.7.swf 880e64ee7614e224660c6616b4bbb1ee5fab8ec9 Diff: https://reviews.apache.org/r/55791/diff/ Testing (updated) --- make check + verification of clipboard functionality Thanks, Jacob Janco
Review Request 55889: Fixes to comments in review 54216.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55889/ --- Review request for mesos. Repository: mesos Description --- Fixes to comments in review 54216. Diffs - src/webui/master/static/js/controllers.js b6364fa636a6dc4b8d314285bec7dc19eb1c9c3b src/webui/master/static/pailer.html 2f48d2393088bd1eceab0f25174c0366cdb41694 Diff: https://reviews.apache.org/r/55889/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 54410: Made the style of the cluster name consistent with others.
> On Dec. 9, 2016, 7:51 p.m., Benjamin Mahler wrote: > > If it looks the same as the other navbar tabs, it seems clickable? At least > > when it is greyed out I wouldn't think I can click it. > > haosdent huang wrote: > Got it, use `btn disabled` to make it grey now. Hey, responding to your review req -> I checked the current styling and it looks consistent for me with no change, alignment/text size and font is the same. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54410/#review158716 --- On Dec. 13, 2016, 7:58 a.m., haosdent huang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54410/ > --- > > (Updated Dec. 13, 2016, 7:58 a.m.) > > > Review request for mesos, Benjamin Mahler and Vinod Kone. > > > Bugs: MESOS-6725 > https://issues.apache.org/jira/browse/MESOS-6725 > > > Repository: mesos > > > Description > --- > > Made the style of the cluster name consistent with other texts in the > navbar. > > > Diffs > - > > src/webui/master/static/index.html a4eda3034906d100a20dcd0a2473112c1edd6b42 > > Diff: https://reviews.apache.org/r/54410/diff/ > > > Testing > --- > > # Before: > >  > > # After: > >  > > > Thanks, > > haosdent huang > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Jan. 19, 2017, 2:45 a.m., Benjamin Mahler wrote:
> > src/master/allocator/mesos/hierarchical.cpp, lines 1273-1280
> > <https://reviews.apache.org/r/51027/diff/11/?file=1589270#file1589270line1273>
> >
> > This change introduces an additional trip through the allocator's queue
> > after the allocation completes and before the delay is called.
> >
> > This would avoid it:
> >
> > ```
> > auto pid = self();
> >
> > // TODO: Use process::loop for the allocation loop.
> > allocate()
> > .onAny([pid]() {
> > delay(allocationInterval, self(), &Self::batch);
> > }
> > ```
> >
> > Not sure if this was intentional or not.
This was unintentional, defer does not need to be called here.
- Jacob
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review162229
---
On Jan. 12, 2017, 6:55 p.m., Jacob Janco wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> ---
>
> (Updated Jan. 12, 2017, 6:55 p.m.)
>
>
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus
> Ma, and Jiang Yan Xu.
>
>
> Bugs: MESOS-6904
> https://issues.apache.org/jira/browse/MESOS-6904
>
>
> Repository: mesos
>
>
> Description
> ---
>
> - Triggered allocations dispatch allocate() only
> if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
> cleared when enqueued allocations are processed.
>
>
> Diffs
> -
>
> src/master/allocator/mesos/hierarchical.hpp
> a6424d624864155e1c87a28a63b784512c5c8722
> src/master/allocator/mesos/hierarchical.cpp
> 91b1ec43940a788459f045ca4a4b82d4e8373bca
>
> Diff: https://reviews.apache.org/r/51027/diff/
>
>
> Testing
> ---
>
> make check with the filters below
>
> Broken tests:
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in
> 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable),
> fix in 51621
>
>
> Thanks,
>
> Jacob Janco
>
>
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Jan. 27, 2017, 2:33 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 9b66c23f26b37c02ed850c06c4518ea99078b02d src/master/allocator/mesos/hierarchical.cpp c2211be7458755aeb91ef078e4bfe92ac474044a Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 52534: Dispatch filter expiration twice.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/ --- (Updated Jan. 27, 2017, 2:34 a.m.) Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - With an asynchronous `batch()` allocation, this ensures that filters do not expire before the next allocation. - This patch should be reverted when allocation occurs on resource recovery. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 9b66c23f26b37c02ed850c06c4518ea99078b02d src/master/allocator/mesos/hierarchical.cpp c2211be7458755aeb91ef078e4bfe92ac474044a Diff: https://reviews.apache.org/r/52534/diff/ Testing --- With https://reviews.apache.org/r/51027/: GTEST_FILTER="-*SmallOfferFilter*" make check -j8 Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 27, 2017, 2:33 a.m.) Review request for mesos and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Jan. 23, 2017, 11:08 p.m., Jiang Yan Xu wrote: > > src/tests/hierarchical_allocator_tests.cpp, line 3765 > > <https://reviews.apache.org/r/51028/diff/6/?file=1590243#file1590243line3765> > > > > Is `s/performing/; performed/?` better? yea I like that better - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review162722 --- On Jan. 27, 2017, 2:33 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Jan. 27, 2017, 2:33 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Jan. 21, 2017, 1:24 a.m., Jiang Yan Xu wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 1314 > > <https://reviews.apache.org/r/51027/diff/11/?file=1589270#file1589270line1314> > > > > We can use the `|=` operator now that it's added. Added, was waiting for that commit =D - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review162529 --- On Jan. 27, 2017, 2:33 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Jan. 27, 2017, 2:33 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > 9b66c23f26b37c02ed850c06c4518ea99078b02d > src/master/allocator/mesos/hierarchical.cpp > c2211be7458755aeb91ef078e4bfe92ac474044a > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check with the filters below > > Broken tests: > - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 > - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in > 51028 > - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 > - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), > fix in 51621 > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Jan. 19, 2017, 3:15 a.m., Benjamin Mahler wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 2994-2998 > > <https://reviews.apache.org/r/51028/diff/6/?file=1590243#file1590243line2994> > > > > This test is already paused, all of these should be run with a paused > > clock, are any of them not? Earlier in the test: // Allow the allocation timer to measure time. Clock::resume(); - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review162237 --- On Jan. 27, 2017, 2:33 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Jan. 27, 2017, 2:33 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Ensure that expected > allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 30, 2017, 11:15 a.m.) Review request for mesos and Jiang Yan Xu. Changes --- Updated review to depend on /r/52534 Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 55852: Fixed MasterAllocatorTest/1.RebalancedForUpdatedWeights.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55852/#review163594 --- Ship it! Ship It! - Jacob Janco On Jan. 23, 2017, 7:08 p.m., Jiang Yan Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55852/ > --- > > (Updated Jan. 23, 2017, 7:08 p.m.) > > > Review request for mesos, Guangya Liu and Jacob Janco. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - This test is broken by the batched allocation change. > > > Diffs > - > > src/tests/master_allocator_tests.cpp > 996762f25453f7a8a5e0b7b97006ee2a603cf8c4 > > Diff: https://reviews.apache.org/r/55852/diff/ > > > Testing > --- > > make check. > > > Thanks, > > Jiang Yan Xu > >
Re: Review Request 55852: Fixed MasterAllocatorTest/1.RebalancedForUpdatedWeights.
> On Jan. 30, 2017, 11:51 p.m., Jacob Janco wrote: > > Ship It! Makes sense, with batched allocation recovery and allocation can be interleaved causing flakiness in this test. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55852/#review163594 --- On Jan. 23, 2017, 7:08 p.m., Jiang Yan Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55852/ > --- > > (Updated Jan. 23, 2017, 7:08 p.m.) > > > Review request for mesos, Guangya Liu and Jacob Janco. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - This test is broken by the batched allocation change. > > > Diffs > - > > src/tests/master_allocator_tests.cpp > 996762f25453f7a8a5e0b7b97006ee2a603cf8c4 > > Diff: https://reviews.apache.org/r/55852/diff/ > > > Testing > --- > > make check. > > > Thanks, > > Jiang Yan Xu > >
Re: Review Request 55852: Fixed MasterAllocatorTest/1.RebalancedForUpdatedWeights.
> On Jan. 30, 2017, 11:51 p.m., Jacob Janco wrote: > > Ship It! > > Jacob Janco wrote: > Makes sense, with batched allocation recovery and allocation can be > interleaved causing flakiness in this test. Passed --gtest_repeat=1000 - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55852/#review163594 --- On Jan. 23, 2017, 7:08 p.m., Jiang Yan Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55852/ > --- > > (Updated Jan. 23, 2017, 7:08 p.m.) > > > Review request for mesos, Guangya Liu and Jacob Janco. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - This test is broken by the batched allocation change. > > > Diffs > - > > src/tests/master_allocator_tests.cpp > 996762f25453f7a8a5e0b7b97006ee2a603cf8c4 > > Diff: https://reviews.apache.org/r/55852/diff/ > > > Testing > --- > > make check. > > > Thanks, > > Jiang Yan Xu > >
Re: Review Request 55893: Fixed OversubscriptionTest.RescindRevocableOfferWithIncreasedRevocable.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55893/#review163604
---
Running at --gtest_repeat=1000 failed with:
../../src/tests/oversubscription_tests.cpp:544: Failure
Value of: resources3
Actual: { cpus(*){REV}:2 }
Expected: resources2
Which is: { cpus(*){REV}:3 }
*** Aborted at 1485821402 (unix time) try "date -d @1485821402" if you are
using GNU date ***
PC: @0x108522f60 testing::UnitTest::AddTestPartResult()
*** SIGSEGV (@0x0) received by PID 13601 (TID 0x7fffcaede3c0) stack trace: ***
@ 0x7fffc229ebba _sigtramp
@0xa (unknown)
@0x108522767 testing::internal::AssertHelper::operator=()
@0x107aed3f1
mesos::internal::tests::OversubscriptionTest_RescindRevocableOfferWithIncreasedRevocable_Test::TestBody()
@0x107ab780a
testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@0x108534877
testing::internal::HandleExceptionsInMethodIfSupported<>()
@0x108534725 testing::Test::Run()
@0x108536278 testing::TestInfo::Run()
@0x1085375e7 testing::TestCase::Run()
@0x10854716c testing::internal::UnitTestImpl::RunAllTests()
@0x108582eda
testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@0x108546b97
testing::internal::HandleExceptionsInMethodIfSupported<>()
@0x108546a68 testing::UnitTest::Run()
@0x107642ee1 RUN_ALL_TESTS()
@0x10763ec1d main
@ 0x7fffc2091255 start
Segmentation fault: 11
- Jacob Janco
On Jan. 24, 2017, 9:52 p.m., Jiang Yan Xu wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55893/
> ---
>
> (Updated Jan. 24, 2017, 9:52 p.m.)
>
>
> Review request for mesos, Benjamin Mahler, Guangya Liu, and Jacob Janco.
>
>
> Bugs: MESOS-6904
> https://issues.apache.org/jira/browse/MESOS-6904
>
>
> Repository: mesos
>
>
> Description
> ---
>
> This is another test fix in the chain.
>
>
> Diffs
> -
>
> src/tests/oversubscription_tests.cpp
> 22ae069ab71c82c6a6e2f5783b13ebd74a9ccf25
>
> Diff: https://reviews.apache.org/r/55893/diff/
>
>
> Testing
> ---
>
> make check.
>
>
> Thanks,
>
> Jiang Yan Xu
>
>
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Jan. 31, 2017, 1:41 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Changes --- Rebase. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 31, 2017, 1:41 a.m.) Review request for mesos and Jiang Yan Xu. Changes --- Rebase. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp e04d1998679fcf022bb3741676a62da8b01ce97c Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Jan. 31, 2017, 1:46 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Changes --- Small fix to batch allocation. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check with the filters below Broken tests: - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028 - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621 Thanks, Jacob Janco
Re: Review Request 52534: Dispatch filter expiration twice.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/ --- (Updated Jan. 31, 2017, 1:47 a.m.) Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan Xu. Changes --- Rebase. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - With an asynchronous `batch()` allocation, this ensures that filters do not expire before the next allocation. - This patch should be reverted when allocation occurs on resource recovery. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 Diff: https://reviews.apache.org/r/52534/diff/ Testing --- With https://reviews.apache.org/r/51027/: GTEST_FILTER="-*SmallOfferFilter*" make check -j8 Thanks, Jacob Janco
Re: Review Request 55874: Added a simple AllocatorBacklog benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55874/#review163611 --- Ship it! Ship It! - Jacob Janco On Jan. 24, 2017, 9:46 p.m., Jiang Yan Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55874/ > --- > > (Updated Jan. 24, 2017, 9:46 p.m.) > > > Review request for mesos, Benjamin Mahler, Jacob Janco, and Zhitao Li. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > To evaluate MESOS-6904. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 > > Diff: https://reviews.apache.org/r/55874/diff/ > > > Testing > --- > > ## Interpretation the benchmark output > > - With the allocation batching it's faster to process all the `reviveOffers` > events after all resources in the cluster have already been allocated. These > allocation runs then don't generate allocations and so their execution should > be identical and the allocation batching reduces the number of them. > - With the allocation batching it takes about the same time to process the > same number of `addSlave` events, no matter if allocations are batched or > not. This is because individual `addSlave` calls trigger only allocation for > that agent while batched allocation allocates all of the accumulated > candidates so far, so the total cost should be the similar. > - It's also faster to process `addFrameworks` with allocation batching but in > this benchmark frameworks are added when no agents are connected, so the > allocation runs should exit very quickly. > > ## Output > > With the patch set: > > ``` > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > Using 1000 agents and 1 frameworks > Added 1 frameworks in 1.638795ms with 1 allocation runs > Added 1000 agents in 799.084178ms with 4 allocation runs > Processed 1 `reviveOffers` calls in 10.030412ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > (909 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > Using 1000 agents and 50 frameworks > Added 50 frameworks in 5.496104ms with 2 allocation runs > Added 1000 agents in 1.210938725secs with 3 allocation runs > Processed 50 `reviveOffers` calls in 129.410729ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > (1420 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > Using 1000 agents and 100 frameworks > Added 100 frameworks in 9.31721ms with 2 allocation runs > Added 1000 agents in 1.064227156secs with 3 allocation runs > Processed 100 `reviveOffers` calls in 229.829276ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > (1410 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > Using 1000 agents and 200 frameworks > Added 200 frameworks in 19.380803ms with 3 allocation runs > Added 1000 agents in 1.472364101secs with 2 allocation runs > Processed 200 `reviveOffers` calls in 419.054445ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > (2007 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > Using 1000 agents and 500 frameworks > Added 500 frameworks in 39.074317ms with 2 allocation runs > Added 1000 agents in 2.561990683secs with 2 allocation runs > Processed 500 `reviveOffers` calls in 754.971627ms with 2 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > (3440 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5 > Using 1000 agents and 1000 frameworks > Added 1000 frameworks in 73.831892ms with 2 allocation runs > Added 1000 agents in 4.395376381secs with 2 allocation runs > Processed 1000 `reviveOffers` calls in 1.660467486secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5
Re: Review Request 52534: Dispatch filter expiration twice.
> On Jan. 31, 2017, 7:39 a.m., Jiang Yan Xu wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 1813-1823 > > <https://reviews.apache.org/r/52534/diff/4/?file=1619670#file1619670line1813> > > > > Actually no need to disambiguate now that we only have one `_expire`. Will have to disambiguate the overloaded `expire()` - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/#review163633 --- On Jan. 31, 2017, 1:47 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52534/ > --- > > (Updated Jan. 31, 2017, 1:47 a.m.) > > > Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan > Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - With an asynchronous `batch()` allocation, > this ensures that filters do not expire > before the next allocation. > - This patch should be reverted when allocation > occurs on resource recovery. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > 2cda3e311ce339d82065d68de83e86439fa192ff > src/master/allocator/mesos/hierarchical.cpp > f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 > > Diff: https://reviews.apache.org/r/52534/diff/ > > > Testing > --- > > With https://reviews.apache.org/r/51027/: > > GTEST_FILTER="-*SmallOfferFilter*" make check -j8 > > > Thanks, > > Jacob Janco > >
Re: Review Request 52534: Dispatch filter expiration twice.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52534/ --- (Updated Jan. 31, 2017, 9:41 p.m.) Review request for mesos, Alexander Rukletsov, Benjamin Mahler, and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - With an asynchronous `batch()` allocation, this ensures that filters do not expire before the next allocation. - This patch should be reverted when allocation occurs on resource recovery. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 Diff: https://reviews.apache.org/r/52534/diff/ Testing --- With https://reviews.apache.org/r/51027/: GTEST_FILTER="-*SmallOfferFilter*" make check -j8 Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 31, 2017, 10:04 p.m.) Review request for mesos and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp e04d1998679fcf022bb3741676a62da8b01ce97c Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Jan. 31, 2017, 10:07 p.m.) Review request for mesos and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp e04d1998679fcf022bb3741676a62da8b01ce97c Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 55893: Fixed OversubscriptionTest.RescindRevocableOfferWithIncreasedRevocable.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55893/#review163723 --- Ship it! Ship It! - Jacob Janco On Jan. 31, 2017, 9:03 a.m., Jiang Yan Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55893/ > --- > > (Updated Jan. 31, 2017, 9:03 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, and Jacob Janco. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > This is another test fix in the chain. > > > Diffs > - > > src/tests/oversubscription_tests.cpp > 22ae069ab71c82c6a6e2f5783b13ebd74a9ccf25 > > Diff: https://reviews.apache.org/r/55893/diff/ > > > Testing > --- > > make check. > > ./bin/mesos-tests.sh --verbose --gtest_repeat=500 > --gtest_filter=*RescindRevocableOfferWithIncreasedRevocable* > > > Thanks, > > Jiang Yan Xu > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Feb. 1, 2017, 10:36 a.m.) Review request for mesos and Jiang Yan Xu. Bugs: MESOS-6904 https://issues.apache.org/jira/browse/MESOS-6904 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Ensure that expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp e04d1998679fcf022bb3741676a62da8b01ce97c Diff: https://reviews.apache.org/r/51028/diff/ Testing --- make check Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Feb. 3, 2017, 10:45 a.m., Benjamin Bannier wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 1263 > > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263> > > > > Could you follow up with a patch so this call is `dispatch`'ed? > > > > Right now, if the `HierarchicalAllocatorProcess` goes away after > > `allocate`, but before the `AnyCallback` finishes, > > `this->allocationInterval` might be referencing garbage. > > Jiang Yan Xu wrote: > `delay` is already dispatched isn't it? > > Benjamin Bannier wrote: > Yes, the `delay`'ed part is safe as it dispatches to a specific process. > The issue here is with the `AnyCallBack` (the full argument to `onAny`) since > it does capture a raw ptr (`this`), but is not dispatched. > > Jiang Yan Xu wrote: > Ah I see what you mean now, thanks! I guess we can copy > `allocationInterval` in. ^I'll submit a patch with the above fix. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review164111 --- On Jan. 31, 2017, 1:46 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Jan. 31, 2017, 1:46 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > 2cda3e311ce339d82065d68de83e86439fa192ff > src/master/allocator/mesos/hierarchical.cpp > f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check with the filters below > > Broken tests: > - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 > - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in > 51028 > - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 > - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), > fix in 51621 > > > Thanks, > > Jacob Janco > >
Review Request 56296: Fix to potential dangling pointer in `batch()`.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56296/ --- Review request for mesos. Repository: mesos Description --- - If `HierarchicalAllocatorProcess` gone after allocate but before `AnyCallback` then `this->allocationInterval` may reference garbage. Diffs - src/master/allocator/mesos/hierarchical.cpp 5f540569043df9d2bb75416c8c36bb4dd7bd68a1 Diff: https://reviews.apache.org/r/56296/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 56296: Fix to potential dangling pointer in `batch()`.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56296/ --- (Updated Feb. 3, 2017, 3:53 p.m.) Review request for mesos, Benjamin Bannier and Jiang Yan Xu. Repository: mesos Description --- - If `HierarchicalAllocatorProcess` gone after allocate but before `AnyCallback` then `this->allocationInterval` may reference garbage. Diffs - src/master/allocator/mesos/hierarchical.cpp 5f540569043df9d2bb75416c8c36bb4dd7bd68a1 Diff: https://reviews.apache.org/r/56296/diff/ Testing (updated) --- make check. Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Feb. 3, 2017, 10:45 a.m., Benjamin Bannier wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 1263 > > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263> > > > > Could you follow up with a patch so this call is `dispatch`'ed? > > > > Right now, if the `HierarchicalAllocatorProcess` goes away after > > `allocate`, but before the `AnyCallback` finishes, > > `this->allocationInterval` might be referencing garbage. > > Jiang Yan Xu wrote: > `delay` is already dispatched isn't it? > > Benjamin Bannier wrote: > Yes, the `delay`'ed part is safe as it dispatches to a specific process. > The issue here is with the `AnyCallBack` (the full argument to `onAny`) since > it does capture a raw ptr (`this`), but is not dispatched. > > Jiang Yan Xu wrote: > Ah I see what you mean now, thanks! I guess we can copy > `allocationInterval` in. > > Jacob Janco wrote: > ^I'll submit a patch with the above fix. https://reviews.apache.org/r/56296 - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review164111 --- On Jan. 31, 2017, 1:46 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Jan. 31, 2017, 1:46 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > 2cda3e311ce339d82065d68de83e86439fa192ff > src/master/allocator/mesos/hierarchical.cpp > f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check with the filters below > > Broken tests: > - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028 > - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028 > - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in > 51028 > - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534 > - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), > fix in 51621 > > > Thanks, > > Jacob Janco > >
Review Request 50268: Target shutdownFramework to associated agents.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50268/ --- Review request for mesos. Repository: mesos Description --- Target shutdownFramework to associated agents. Diffs - src/master/master.cpp 78a8889313179b4509a6657e6c61d9f6d3bb99c0 src/slave/slave.cpp da643e6e50b2f313705d2f862c961291aa5d2f22 Diff: https://reviews.apache.org/r/50268/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 50268: Target shutdownFramework to associated agents.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50268/ --- (Updated July 20, 2016, 10:08 p.m.) Review request for mesos. Repository: mesos Description --- Target shutdownFramework to associated agents. Diffs - src/master/master.cpp 78a8889313179b4509a6657e6c61d9f6d3bb99c0 src/slave/slave.cpp da643e6e50b2f313705d2f862c961291aa5d2f22 Diff: https://reviews.apache.org/r/50268/diff/ Testing (updated) --- 5 frameworks on 1 agent with 2 agents in cluster pre: ShutDownFrameworkMessage sent to both agents. post: ShutdownFrameworkMessage sent to agent with executors associated with framework. Thanks, Jacob Janco
Re: Review Request 50268: Target shutdownFramework to associated agents.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50268/ --- (Updated July 20, 2016, 10:09 p.m.) Review request for mesos. Bugs: MESOS-5874 https://issues.apache.org/jira/browse/MESOS-5874 Repository: mesos Description --- Target shutdownFramework to associated agents. Diffs - src/master/master.cpp 78a8889313179b4509a6657e6c61d9f6d3bb99c0 src/slave/slave.cpp da643e6e50b2f313705d2f862c961291aa5d2f22 Diff: https://reviews.apache.org/r/50268/diff/ Testing --- 5 frameworks on 1 agent with 2 agents in cluster pre: ShutDownFrameworkMessage sent to both agents. post: ShutdownFrameworkMessage sent to agent with executors associated with framework. Thanks, Jacob Janco
Re: Review Request 49616: Add suppression benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/ --- (Updated July 26, 2016, 5:29 a.m.) Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. Bugs: MESOS-5781 https://issues.apache.org/jira/browse/MESOS-5781 Repository: mesos Description --- - Useful for high framework clusters which carry many suppressed frameworks that are considered during allocation. Diffs (updated) - docs/contributors.yaml b8ecdad2591ba5ee0de50e970b7276587db22270 src/tests/gc_tests.cpp 11a31dbb8d12bfb8a7f10c8eb87154289ac96f62 src/tests/hierarchical_allocator_tests.cpp 0498cd5e54b0e4b87a767585a77699653aa52179 Diff: https://reviews.apache.org/r/49616/diff/ Testing --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check Thanks, Jacob Janco
Re: Review Request 49616: Add suppression benchmark.
> On July 23, 2016, 1:37 a.m., Benjamin Mahler wrote:
> > Thanks Jacob! I left some comments here around style and readability of the
> > code. It looks like much of this was copied and so I realize that you
> > weren't the one to introduce many of these patterns. Nonetheless, would be
> > great to simplify this!
Thanks for the review Ben! Your comments are definitely apropos and very
helpful for my own understanding.
> On July 23, 2016, 1:37 a.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp, lines 3579-3581
> > <https://reviews.apache.org/r/49616/diff/4/?file=1439839#file1439839line3579>
> >
> > I'm confused by this comment, why is this relevant if the clock is
> > paused?
Advancing the clock by the default allocation_interval triggers the batch
allocate, I agree this is not relevant.
> On July 23, 2016, 1:37 a.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp, lines 3649-3652
> > <https://reviews.apache.org/r/49616/diff/4/?file=1439839#file1439839line3649>
> >
> > Per my comment above, it seems much easier to understand this code if
> > we just loop on the number of allocations we expect:
> >
> > ```
> > size_t allocationsCount = 5;
> > size_t suppressCount = 0;
> >
> > for (size_t i = 0; i < allocationsCount; ++i) {
> > ...
> >
> > // Suppress another batch of frameworks.
> > for (size_t j = 0; j < frameworkCount / allocationsCount; ++j) {
> > allocator->suppressOffers(frameworks[suppressCount].id());
> > ++suppressCount;
> > }
> >
> > ...
> > }
> > ```
I agree this is easier to understand and simpler if we loop by the expected
counts.
- Jacob
-------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49616/#review143308
---
On July 26, 2016, 5:29 a.m., Jacob Janco wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49616/
> ---
>
> (Updated July 26, 2016, 5:29 a.m.)
>
>
> Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu.
>
>
> Bugs: MESOS-5781
> https://issues.apache.org/jira/browse/MESOS-5781
>
>
> Repository: mesos
>
>
> Description
> ---
>
> - Useful for high framework clusters which carry
> many suppressed frameworks that are considered
> during allocation.
>
>
> Diffs
> -
>
> docs/contributors.yaml b8ecdad2591ba5ee0de50e970b7276587db22270
> src/tests/gc_tests.cpp 11a31dbb8d12bfb8a7f10c8eb87154289ac96f62
> src/tests/hierarchical_allocator_tests.cpp
> 0498cd5e54b0e4b87a767585a77699653aa52179
>
> Diff: https://reviews.apache.org/r/49616/diff/
>
>
> Testing
> ---
>
> MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check
>
>
> Thanks,
>
> Jacob Janco
>
>
Re: Review Request 49616: Add suppression benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/ --- (Updated July 26, 2016, 7:03 a.m.) Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. Bugs: MESOS-5781 https://issues.apache.org/jira/browse/MESOS-5781 Repository: mesos Description --- - Useful for high framework clusters which carry many suppressed frameworks that are considered during allocation. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp bb6947fcecb5b78047e98d10fc1278c612a69548 Diff: https://reviews.apache.org/r/49616/diff/ Testing --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check Thanks, Jacob Janco
Re: Review Request 49616: Add suppression benchmark.
> On July 26, 2016, 6 a.m., Guangya Liu wrote: > > src/tests/hierarchical_allocator_tests.cpp, line 3637 > > <https://reviews.apache.org/r/49616/diff/5/?file=1452185#file1452185line3637> > > > > Add the elapse time of adding slave here, an example here > > https://github.com/apache/mesos/blob/master/src/tests/hierarchical_allocator_tests.cpp#L3714 Collapsed this down to adding slaves rather than tracking for adding frameworks as in the scenario the frameworks are added without resources available. Instead why not track frameworks/adding slaves in with a portion of resources used? What do you think? - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/#review143484 --- On July 26, 2016, 7:03 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49616/ > --- > > (Updated July 26, 2016, 7:03 a.m.) > > > Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. > > > Bugs: MESOS-5781 > https://issues.apache.org/jira/browse/MESOS-5781 > > > Repository: mesos > > > Description > --- > > - Useful for high framework clusters which carry > many suppressed frameworks that are considered > during allocation. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > bb6947fcecb5b78047e98d10fc1278c612a69548 > > Diff: https://reviews.apache.org/r/49616/diff/ > > > Testing > --- > > MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 49616: Add suppression benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/ --- (Updated July 26, 2016, 7:28 a.m.) Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. Bugs: MESOS-5781 https://issues.apache.org/jira/browse/MESOS-5781 Repository: mesos Description --- - Useful for high framework clusters which carry many suppressed frameworks that are considered during allocation. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp bb6947fcecb5b78047e98d10fc1278c612a69548 Diff: https://reviews.apache.org/r/49616/diff/ Testing --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check Thanks, Jacob Janco
Re: Review Request 49616: Add suppression benchmark.
> On July 26, 2016, 6 a.m., Guangya Liu wrote: > > src/tests/hierarchical_allocator_tests.cpp, line 3637 > > <https://reviews.apache.org/r/49616/diff/5/?file=1452185#file1452185line3637> > > > > Add the elapse time of adding slave here, an example here > > https://github.com/apache/mesos/blob/master/src/tests/hierarchical_allocator_tests.cpp#L3714 > > Jacob Janco wrote: > Collapsed this down to adding slaves rather than tracking for adding > frameworks as in the scenario the frameworks are added without resources > available. Instead why not track frameworks/adding slaves in with a portion > of resources used? What do you think? > > Guangya Liu wrote: > Thanks Jacob, I think that tracking those values separatelly maybe > better. Even though adding framework will have no resources, but this can > calculate the elapse time of adding frameworks and the time of allocator > caused if there are no agents. Sounds good, adding. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/#review143484 --- On July 26, 2016, 7:28 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49616/ > --- > > (Updated July 26, 2016, 7:28 a.m.) > > > Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. > > > Bugs: MESOS-5781 > https://issues.apache.org/jira/browse/MESOS-5781 > > > Repository: mesos > > > Description > --- > > - Useful for high framework clusters which carry > many suppressed frameworks that are considered > during allocation. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > bb6947fcecb5b78047e98d10fc1278c612a69548 > > Diff: https://reviews.apache.org/r/49616/diff/ > > > Testing > --- > > MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 49616: Add suppression benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/ --- (Updated July 28, 2016, 8:42 p.m.) Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. Bugs: MESOS-5781 https://issues.apache.org/jira/browse/MESOS-5781 Repository: mesos Description --- - Useful for high framework clusters which carry many suppressed frameworks that are considered during allocation. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp bb6947fcecb5b78047e98d10fc1278c612a69548 Diff: https://reviews.apache.org/r/49616/diff/ Testing (updated) --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check ... [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12 Using 5000 agents and 500 frameworks Added 500 frameworks in 9300us Added 5000 agents in 15.881391secs allocate() took 15.537025secs to make 5000 offers with 100 out of 500 frameworks suppressing offers allocate() took 15.175028secs to make 5000 offers with 200 out of 500 frameworks suppressing offers allocate() took 15.554843secs to make 5000 offers with 300 out of 500 frameworks suppressing offers allocate() took 15.740833secs to make 5000 offers with 400 out of 500 frameworks suppressing offers allocate() took 1.311314secs to make 0 offers with 500 out of 500 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12 (93269 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13 Using 5000 agents and 1000 frameworks Added 1000 frameworks in 17917us Added 5000 agents in 31.450037secs allocate() took 31.807624secs to make 5000 offers with 200 out of 1000 frameworks suppressing offers allocate() took 32.039667secs to make 5000 offers with 400 out of 1000 frameworks suppressing offers allocate() took 32.333716secs to make 5000 offers with 600 out of 1000 frameworks suppressing offers allocate() took 32.938456secs to make 5000 offers with 800 out of 1000 frameworks suppressing offers allocate() took 2.69174secs to make 0 offers with 1000 out of 1000 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13 (178827 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14 Using 5000 agents and 3000 frameworks Added 3000 frameworks in 55821us Added 5000 agents in 1.6420837333mins allocate() took 1.6388713167mins to make 5000 offers with 600 out of 3000 frameworks suppressing offers allocate() took 1.6645890833mins to make 5000 offers with 1200 out of 3000 frameworks suppressing offers allocate() took 1.6524411667mins to make 5000 offers with 1800 out of 3000 frameworks suppressing offers allocate() took 1.71699065mins to make 5000 offers with 2400 out of 3000 frameworks suppressing offers allocate() took 8.371823secs to make 0 offers with 3000 out of 3000 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14 (528494 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/15 Using 5000 agents and 6000 frameworks Added 6000 frameworks in 110493us Added 5000 agents in 3.342676mins allocate() took 3.6809198667mins to make 5000 offers with 1200 out of 6000 frameworks suppressing offers allocate() took 3.6710132333mins to make 5000 offers with 2400 out of 6000 frameworks suppressing offers allocate() took 3.6804925167mins to make 5000 offers with 3600 out of 6000 frameworks suppressing offers allocate() took 3.7197498333mins to make 5000 offers with 4800 out of 6000 frameworks suppressing offers allocate() took 17.318823secs to make 0 offers with 6000 out of 6000 frameworks suppressing offers ... Thanks, Jacob Janco
Re: Review Request 49616: Add suppression benchmark.
> On July 28, 2016, 4:02 p.m., Jiang Yan Xu wrote:
> > src/tests/hierarchical_allocator_tests.cpp, line 3813
> > <https://reviews.apache.org/r/49616/diff/7/?file=1452440#file1452440line3813>
> >
> > The asymmetry between the string typed `agentTotal` and the `Resources`
> > typed `allocation` looks jarring to me, how about just:
> >
> > ```
> > SlaveInfo agent =
> > createSlaveInfo("cpus:24;mem:4096;disk:4096;ports:[31000-32000]");
> >
> > // Then later,
> >
> > agents.push_back(agent);
> > ```
Need to call createSlaveInfo in the loop to generate unique slaveIds on each
pass.
- Jacob
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49616/#review143901
---
On July 28, 2016, 8:42 p.m., Jacob Janco wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49616/
> ---
>
> (Updated July 28, 2016, 8:42 p.m.)
>
>
> Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu.
>
>
> Bugs: MESOS-5781
> https://issues.apache.org/jira/browse/MESOS-5781
>
>
> Repository: mesos
>
>
> Description
> ---
>
> - Useful for high framework clusters which carry
> many suppressed frameworks that are considered
> during allocation.
>
>
> Diffs
> -
>
> src/tests/hierarchical_allocator_tests.cpp
> bb6947fcecb5b78047e98d10fc1278c612a69548
>
> Diff: https://reviews.apache.org/r/49616/diff/
>
>
> Testing
> ---
>
> MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check
>
> ...
> [ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12
> Using 5000 agents and 500 frameworks
> Added 500 frameworks in 9300us
> Added 5000 agents in 15.881391secs
> allocate() took 15.537025secs to make 5000 offers with 100 out of 500
> frameworks suppressing offers
> allocate() took 15.175028secs to make 5000 offers with 200 out of 500
> frameworks suppressing offers
> allocate() took 15.554843secs to make 5000 offers with 300 out of 500
> frameworks suppressing offers
> allocate() took 15.740833secs to make 5000 offers with 400 out of 500
> frameworks suppressing offers
> allocate() took 1.311314secs to make 0 offers with 500 out of 500 frameworks
> suppressing offers
> [ OK ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12
> (93269 ms)
> [ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13
> Using 5000 agents and 1000 frameworks
> Added 1000 frameworks in 17917us
> Added 5000 agents in 31.450037secs
> allocate() took 31.807624secs to make 5000 offers with 200 out of 1000
> frameworks suppressing offers
> allocate() took 32.039667secs to make 5000 offers with 400 out of 1000
> frameworks suppressing offers
> allocate() took 32.333716secs to make 5000 offers with 600 out of 1000
> frameworks suppressing offers
> allocate() took 32.938456secs to make 5000 offers with 800 out of 1000
> frameworks suppressing offers
> allocate() took 2.69174secs to make 0 offers with 1000 out of 1000 frameworks
> suppressing offers
> [ OK ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13
> (178827 ms)
> [ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14
> Using 5000 agents and 3000 frameworks
> Added 3000 frameworks in 55821us
> Added 5000 agents in 1.6420837333mins
> allocate() took 1.6388713167mins to make 5000 offers with 600 out of 3000
> frameworks suppressing offers
> allocate() took 1.6645890833mins to make 5000 offers with 1200 out of
> 3000 frameworks suppressing offers
> allocate() took 1.6524411667mins to make 5000 offers with 1800 out of
> 3000 frameworks suppressing offers
> allocate() took 1.71699065mins to make 5000 offers with 2400 out of 3000
> frameworks suppressing offers
> allocate() took 8.371823secs to make 0 offers with 3000 out of 3000
> frameworks suppressing offers
> [ OK ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14
> (528494 ms)
> [ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/15
> Using 5000 agents and 6000 frameworks
> Added 6000 frameworks in 110493us
> Added 5000 agents in 3.
Re: Review Request 49616: Add suppression benchmark.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49616/ --- (Updated July 29, 2016, 5:05 p.m.) Review request for mesos, James Peach, Joris Van Remoortere, and Jiang Yan Xu. Bugs: MESOS-5781 https://issues.apache.org/jira/browse/MESOS-5781 Repository: mesos Description --- - Useful for high framework clusters which carry many suppressed frameworks that are considered during allocation. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp acea1277bf4980743a0b875af5c2861cffe80ab5 Diff: https://reviews.apache.org/r/49616/diff/ Testing --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.SuppressOffers*" make check ... [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12 Using 5000 agents and 500 frameworks Added 500 frameworks in 9300us Added 5000 agents in 15.881391secs allocate() took 15.537025secs to make 5000 offers with 100 out of 500 frameworks suppressing offers allocate() took 15.175028secs to make 5000 offers with 200 out of 500 frameworks suppressing offers allocate() took 15.554843secs to make 5000 offers with 300 out of 500 frameworks suppressing offers allocate() took 15.740833secs to make 5000 offers with 400 out of 500 frameworks suppressing offers allocate() took 1.311314secs to make 0 offers with 500 out of 500 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/12 (93269 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13 Using 5000 agents and 1000 frameworks Added 1000 frameworks in 17917us Added 5000 agents in 31.450037secs allocate() took 31.807624secs to make 5000 offers with 200 out of 1000 frameworks suppressing offers allocate() took 32.039667secs to make 5000 offers with 400 out of 1000 frameworks suppressing offers allocate() took 32.333716secs to make 5000 offers with 600 out of 1000 frameworks suppressing offers allocate() took 32.938456secs to make 5000 offers with 800 out of 1000 frameworks suppressing offers allocate() took 2.69174secs to make 0 offers with 1000 out of 1000 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/13 (178827 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14 Using 5000 agents and 3000 frameworks Added 3000 frameworks in 55821us Added 5000 agents in 1.6420837333mins allocate() took 1.6388713167mins to make 5000 offers with 600 out of 3000 frameworks suppressing offers allocate() took 1.6645890833mins to make 5000 offers with 1200 out of 3000 frameworks suppressing offers allocate() took 1.6524411667mins to make 5000 offers with 1800 out of 3000 frameworks suppressing offers allocate() took 1.71699065mins to make 5000 offers with 2400 out of 3000 frameworks suppressing offers allocate() took 8.371823secs to make 0 offers with 3000 out of 3000 frameworks suppressing offers [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/14 (528494 ms) [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/15 Using 5000 agents and 6000 frameworks Added 6000 frameworks in 110493us Added 5000 agents in 3.342676mins allocate() took 3.6809198667mins to make 5000 offers with 1200 out of 6000 frameworks suppressing offers allocate() took 3.6710132333mins to make 5000 offers with 2400 out of 6000 frameworks suppressing offers allocate() took 3.6804925167mins to make 5000 offers with 3600 out of 6000 frameworks suppressing offers allocate() took 3.7197498333mins to make 5000 offers with 4800 out of 6000 frameworks suppressing offers allocate() took 17.318823secs to make 0 offers with 6000 out of 6000 frameworks suppressing offers ... Thanks, Jacob Janco
Re: Review Request 49617: Add benchmark for failover of many frameworks.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49617/ --- (Updated Aug. 17, 2016, 2:17 a.m.) Review request for mesos, Joris Van Remoortere and Jiang Yan Xu. Bugs: MESOS-5780 https://issues.apache.org/jira/browse/MESOS-5780 Repository: mesos Description --- - This benchmark measures latency to stability of the allocator following disconnection and reconnection of all frameworks. - In this scenario, frameworks are offered resources and suppressed in batches. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp cbed333f497016fe2811f755028796012b41db77 Diff: https://reviews.apache.org/r/49617/diff/ Testing (updated) --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.FrameworkFailover*" make check Sample Output: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23 Using 1 agents and 6000 frameworks Added 6000 frameworks in 113410us Added 1 agents in 6.8398066333mins allocator settled after 3.286837mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23 (609255 ms)[ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24 Using 2 agents and 1 frameworks Added 1 frameworks in 190us Added 2 agents in 4.752954secs allocator settled after 7us [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24 (6332 ms) Thanks, Jacob Janco
Re: Review Request 49617: Add benchmark for failover of many frameworks.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49617/ --- (Updated Aug. 17, 2016, 2:26 a.m.) Review request for mesos, Joris Van Remoortere and Jiang Yan Xu. Bugs: MESOS-5780 https://issues.apache.org/jira/browse/MESOS-5780 Repository: mesos Description --- - This benchmark measures latency to stability of the allocator following disconnection and reconnection of all frameworks. - In this scenario, frameworks are offered resources and suppressed in batches. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp cbed333f497016fe2811f755028796012b41db77 Diff: https://reviews.apache.org/r/49617/diff/ Testing --- MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.FrameworkFailover*" make check Sample Output: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23 Using 1 agents and 6000 frameworks Added 6000 frameworks in 113410us Added 1 agents in 6.8398066333mins allocator settled after 3.286837mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23 (609255 ms)[ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24 Using 2 agents and 1 frameworks Added 1 frameworks in 190us Added 2 agents in 4.752954secs allocator settled after 7us [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24 (6332 ms) Thanks, Jacob Janco
Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- Review request for mesos. Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- Thanks, Jacob Janco
Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- Review request for mesos. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Settle the clock between these events in tests sensitive to this to ensure expected allocations occur. Diffs - src/tests/hierarchical_allocator_tests.cpp cbed333f497016fe2811f755028796012b41db77 Diff: https://reviews.apache.org/r/51028/diff/ Testing --- Thanks, Jacob Janco
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/ --- (Updated Aug. 17, 2016, 2:32 a.m.) Review request for mesos. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Per MESOS-3157, if cluster events with the possibility of triggering an allocation occur rapidly and test assertions depend on gleaning information from assumed order, it is likely they will fail due to lack of parity between event and actual allocation. Settle the clock between these events in tests sensitive to this to ensure expected allocations occur. Diffs (updated) - src/tests/hierarchical_allocator_tests.cpp cbed333f497016fe2811f755028796012b41db77 Diff: https://reviews.apache.org/r/51028/diff/ Testing (updated) --- make check Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 17, 2016, 3:46 a.m.) Review request for mesos. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing (updated) --- make check note: check without filters depends on https://reviews.apache.org/r/51028 Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 17, 2016, 3:51 a.m.) Review request for mesos. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing (updated) --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 17, 2016, 6:12 p.m.) Review request for mesos, Benjamin Mahler, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - carrying over work from https://reviews.apache.org/r/41658/ and added the previous reviewers Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 23, 2016, 8:24 a.m.) Review request for . Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 23, 2016, 8:24 a.m.) Review request for . Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 23, 2016, 8:32 a.m.) Review request for mesos, Benjamin Mahler, James Peach, Klaus Ma, and Jiang Yan Xu. Changes --- Adding back reviewers and dependencies. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. - Carrying over work from https://reviews.apache.org/r/41658/ and added the previous reviewers - Specifically, this patch introduces the boolean flag pendingAllocation, which when set on event triggered allocations, will prevent additional no-op allocations: the flag is cleared when the enqueued allocation is processed, subsequent event triggered allocations will update a set of allocation candidates rather than dispatching an additional allocate(). Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Aug. 18, 2016, 12:38 a.m., Guangya Liu wrote: > > Can you please also clarify some difference with > > https://reviews.apache.org/r/41658/, as here the work is carry from > > https://reviews.apache.org/r/41658/ , I think that this is more simple and > > clear compared with counter in https://reviews.apache.org/r/41658/ with a > > boolean here. > > Jiang Yan Xu wrote: > This is exactly a rework based on comments in > https://reviews.apache.org/r/41658/. > > Guangya Liu wrote: > Yes, but as the `Description` part mentioned > https://reviews.apache.org/r/41658/ , so here it would be great to list some > difference with https://reviews.apache.org/r/41658/ , and the most important > thing is with the `boolean` value here, we can avoid some no-op allocate > operations. Added. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review146058 ------- On Aug. 23, 2016, 8:32 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Aug. 23, 2016, 8:32 a.m.) > > > Review request for mesos, Benjamin Mahler, James Peach, Klaus Ma, and Jiang > Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > - Carrying over work from https://reviews.apache.org/r/41658/ and added the > previous reviewers > - Specifically, this patch introduces the boolean flag pendingAllocation, > which when set on event > triggered allocations, will prevent additional no-op allocations: the flag > is cleared when > the enqueued allocation is processed, subsequent event triggered > allocations will update a set > of allocation candidates rather than dispatching an additional allocate(). > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 > src/master/allocator/mesos/hierarchical.cpp > 234ef98529964a0b6d3f132426a6c8ccbb1263ee > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Aug. 26, 2016, 6:24 a.m., Klaus Ma wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 275 > > <https://reviews.apache.org/r/51027/diff/2/?file=1481820#file1481820line275> > > > > I think we should use `insert(...)` instead of `=`; if the following > > event in the queue, are we still going to get the correct result? > > ``` > > batch -> addFramework(f1) -> addFramework(f2) > > ``` insert in a loop? - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review146919 --- On Aug. 23, 2016, 8:49 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Aug. 23, 2016, 8:49 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > - Carrying over work from https://reviews.apache.org/r/41658/ and added the > previous reviewers > - Specifically, this patch introduces the boolean flag pendingAllocation, > which when set on event > triggered allocations, will prevent additional no-op allocations: the flag > is cleared when > the enqueued allocation is processed, subsequent event triggered > allocations will update a set > of allocation candidates rather than dispatching an additional allocate(). > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 > src/master/allocator/mesos/hierarchical.cpp > 234ef98529964a0b6d3f132426a6c8ccbb1263ee > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 10000 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 30, 2016, 4:53 p.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. - Carrying over work from https://reviews.apache.org/r/41658/ and added the previous reviewers - Specifically, this patch introduces the boolean flag pendingAllocation, which when set on event triggered allocations, will prevent additional no-op allocations: the flag is cleared when the enqueued allocation is processed, subsequent event triggered allocations will update a set of allocation candidates rather than dispatching an additional allocate(). Diffs - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Aug. 30, 2016, 4:50 p.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 src/master/allocator/mesos/hierarchical.cpp 234ef98529964a0b6d3f132426a6c8ccbb1263ee Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Aug. 26, 2016, 6:24 a.m., Klaus Ma wrote: > > src/master/allocator/mesos/hierarchical.cpp, line 275 > > <https://reviews.apache.org/r/51027/diff/2/?file=1481820#file1481820line275> > > > > I think we should use `insert(...)` instead of `=`; if the following > > event in the queue, are we still going to get the correct result? > > ``` > > batch -> addFramework(f1) -> addFramework(f2) > > ``` > > Jacob Janco wrote: > insert in a loop? > > Klaus Ma wrote: > No; if multiple frameworks register at almost the same time, will there > be multiple addFramework events pending in the queue? Did not get a chance to > test it, but did we consider such kind of race condition? If I understand this correctly: batch -> addFramework(f1) -> addFramework(f2) will ensure an allocate() at batch over all known slaves. If the two addFramework events come in close succession then the queue should look like 1: addFramework(f1) -> addFramework(f2) 2: addFramework(f2) -> allocate() 3: allocate() There will be a pendingAllocation set for the first event with the dispatched allocate() after addFramework(f2). I'm not sure what the race condition would be as we can have many addFramework events queued. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review146919 --- On Aug. 30, 2016, 4:53 p.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Aug. 30, 2016, 4:53 p.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > > - Carrying over work from > https://reviews.apache.org/r/41658/ and added > the previous reviewers > - Specifically, this patch introduces the boolean > flag pendingAllocation, which when set on event > triggered allocations, will prevent additional > no-op allocations: the flag is cleared when the > enqueued allocation is processed, subsequent > event triggered allocations will update a set > of allocation candidates rather than dispatching > an additional allocate(). > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > bdbc6d3b5b959990538f4e3b7b1a3b031d9aea05 > src/master/allocator/mesos/hierarchical.cpp > 234ef98529964a0b6d3f132426a6c8ccbb1263ee > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/ --- (Updated Sept. 3, 2016, 6:05 a.m.) Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu. Bugs: MESOS-3157 https://issues.apache.org/jira/browse/MESOS-3157 Repository: mesos Description (updated) --- - Triggered allocations dispatch allocate() only if there is no pending allocation in the queue. - Allocation candidates are accumulated and only cleared when enqueued allocations are processed. - Batched allocations are handled synchronously. Diffs (updated) - src/master/allocator/mesos/hierarchical.hpp dd07ed221d2c1755d2478369641ffdc46ecc4471 src/master/allocator/mesos/hierarchical.cpp 9e5db2196c6a541dc1208ba8b9f13ef9a518bcc4 Diff: https://reviews.apache.org/r/51027/diff/ Testing --- make check note: check without filters depends on https://reviews.apache.org/r/51028 With new benchmark https://reviews.apache.org/r/49617: Sample output without 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 57251us Added 1 agents in 3.2134535333mins allocator settled after 1.6123603833mins [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (290578 ms) Sample output with 51027: [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 Using 1 agents and 3000 frameworks Added 3000 frameworks in 39817us Added 1 agents in 3.2286054167mins allocator settled after 25.525654secs [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 (220137 ms) Thanks, Jacob Janco
Re: Review Request 49617: Add benchmark for failover of many frameworks.
> On Aug. 24, 2016, 9:12 a.m., Guangya Liu wrote:
> > src/tests/hierarchical_allocator_tests.cpp, line 4030
> > <https://reviews.apache.org/r/49617/diff/7/?file=1476268#file1476268line4030>
> >
> > How about adding `ports` resources to the allocation here.
> >
> > ```
> > // Each agent has a portion of it's resources allocated to a single
> > // framework. We round-robin through the frameworks when allocating.
> > Resources allocation =
> > Resources::parse("cpus:16;mem:1024;disk:1024").get();
> >
> > Try<::mesos::Value::Ranges> ranges = fragment(createRange(31000,
> > 32000), 16);
> > ASSERT_SOME(ranges);
> > ASSERT_EQ(16, ranges->range_size());
> >
> > allocation += createPorts(ranges.get());
> > ```
I took this out to simplify the test.
- Jacob
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49617/#review146625
---
On Aug. 17, 2016, 2:26 a.m., Jacob Janco wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49617/
> ---
>
> (Updated Aug. 17, 2016, 2:26 a.m.)
>
>
> Review request for mesos, Joris Van Remoortere and Jiang Yan Xu.
>
>
> Bugs: MESOS-5780
> https://issues.apache.org/jira/browse/MESOS-5780
>
>
> Repository: mesos
>
>
> Description
> ---
>
> - This benchmark measures latency to stability of
> the allocator following disconnection and
> reconnection of all frameworks.
> - In this scenario, frameworks are offered resources
> and suppressed in batches.
>
>
> Diffs
> -
>
> src/tests/hierarchical_allocator_tests.cpp
> cbed333f497016fe2811f755028796012b41db77
>
> Diff: https://reviews.apache.org/r/49617/diff/
>
>
> Testing
> ---
>
> MESOS_BENCHMARK=1 GTEST_FILTER="*BENCHMARK_Test.FrameworkFailover*" make check
>
> Sample Output:
> [ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23
> Using 1 agents and 6000 frameworks
> Added 6000 frameworks in 113410us
> Added 1 agents in 6.8398066333mins
> allocator settled after 3.286837mins
> [ OK ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/23
> (609255 ms)[ RUN ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24
> Using 2 agents and 1 frameworks
> Added 1 frameworks in 190us
> Added 2 agents in 4.752954secs
> allocator settled after 7us
> [ OK ]
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/24
> (6332 ms)
>
>
> Thanks,
>
> Jacob Janco
>
>
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Sept. 3, 2016, 6:38 a.m., Guangya Liu wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 1226-1229 > > <https://reviews.apache.org/r/51027/diff/4/?file=1490734#file1490734line1226> > > > > Just nit, I think that we should separate the `_allocate()` and > > `_deallocate()` to use different time elapse. But as the original logic is > > also putting the `_deallocate()` logic in `_allocate()`, so I think that it > > is OK to keep such logic but may need an update if we want to make the time > > more accurate for diffrent actions: `_allocate()` and `_deallocate()`. > > > > ``` > > stopwatch.start(); > > > > metrics.allocation_run.start(); > > > > _allocate(); > > > > metrics.allocation_run.stop(); > > > > stopwatch.stop(); > > > > VLOG(1) << "Performed allocation for " << allocationCandidates.size() > > << " agents in " << stopwatch.elapsed(); > > > > stopwatch.start(); > > > > _deallocate(); > > > > stopwatch.stop(); > > > > VLOG(1) << "Performed deallocation for " << allocationCandidates.size() > > << " agents in " << stopwatch.elapsed(); > > ``` Yea, I think preserving the original behavior was the idea here so the metrics reflect that. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review147718 --- On Sept. 3, 2016, 6:05 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Sept. 3, 2016, 6:05 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > dd07ed221d2c1755d2478369641ffdc46ecc4471 > src/master/allocator/mesos/hierarchical.cpp > 9e5db2196c6a541dc1208ba8b9f13ef9a518bcc4 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Sept. 3, 2016, 9:21 a.m., Guangya Liu wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 484-485 > > <https://reviews.apache.org/r/51027/diff/4/?file=1490734#file1490734line484> > > > > I did some test with benchmark test of > > `SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0` > > and it failed as following: > > > > ``` > > ../../src/tests/hierarchical_allocator_tests.cpp:3459: Failure > > Value of: offerCallbacks.load() > > Actual: 5 > > Expected: slaveCount > > Which is: 1000 > > [ FAILED ] > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0, > > where GetParam() = (1000, 1) (497 ms) > > [--] 1 test from > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test (498 ms total) > > > > [--] Global test environment tear-down > > [==] 1 test from 1 test case ran. (512 ms total) > > [ PASSED ] 0 tests. > > [ FAILED ] 1 test, listed below: > > [ FAILED ] > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0, > > where GetParam() = (1000, 1) > > > > 1 FAILED TEST > > ``` > > > > The reason is that we cannot make sure all of the `_allocate()` > > finished after `addSlave` finished. > > > > Shall we do a `while` loop in the benchmark to wait till all > > allocations are got? > > Guangya Liu wrote: > Perhaps we can simply remove the `ASSERT_EQ` for offer and agent count in > `HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave` as this patch will > not trigger `allocate()` synchronously, so we cannot make sure the agent > count is always same as offer count. Good catch, I'll run the benchmarks and update https://reviews.apache.org/r/51028/ - Jacob ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review147722 --- On Sept. 3, 2016, 6:05 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Sept. 3, 2016, 6:05 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > dd07ed221d2c1755d2478369641ffdc46ecc4471 > src/master/allocator/mesos/hierarchical.cpp > 9e5db2196c6a541dc1208ba8b9f13ef9a518bcc4 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51027: Track allocation candidates to bound allocator.
> On Sept. 3, 2016, 9:21 a.m., Guangya Liu wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 484-485 > > <https://reviews.apache.org/r/51027/diff/4/?file=1490734#file1490734line484> > > > > I did some test with benchmark test of > > `SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0` > > and it failed as following: > > > > ``` > > ../../src/tests/hierarchical_allocator_tests.cpp:3459: Failure > > Value of: offerCallbacks.load() > > Actual: 5 > > Expected: slaveCount > > Which is: 1000 > > [ FAILED ] > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0, > > where GetParam() = (1000, 1) (497 ms) > > [--] 1 test from > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test (498 ms total) > > > > [--] Global test environment tear-down > > [==] 1 test from 1 test case ran. (512 ms total) > > [ PASSED ] 0 tests. > > [ FAILED ] 1 test, listed below: > > [ FAILED ] > > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/0, > > where GetParam() = (1000, 1) > > > > 1 FAILED TEST > > ``` > > > > The reason is that we cannot make sure all of the `_allocate()` > > finished after `addSlave` finished. > > > > Shall we do a `while` loop in the benchmark to wait till all > > allocations are got? > > Guangya Liu wrote: > Perhaps we can simply remove the `ASSERT_EQ` for offer and agent count in > `HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave` as this patch will > not trigger `allocate()` synchronously, so we cannot make sure the agent > count is always same as offer count. > > Jacob Janco wrote: > Good catch, I'll run the benchmarks and update > https://reviews.apache.org/r/51028/ I updated that review if you could take a look. Thanks! - Jacob ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review147722 --- On Sept. 3, 2016, 6:05 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > --- > > (Updated Sept. 3, 2016, 6:05 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > > Diffs > - > > src/master/allocator/mesos/hierarchical.hpp > dd07ed221d2c1755d2478369641ffdc46ecc4471 > src/master/allocator/mesos/hierarchical.cpp > 9e5db2196c6a541dc1208ba8b9f13ef9a518bcc4 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > --- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 1 agents in 3.2134535333mins > allocator settled after 1.6123603833mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 1 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 1 agents in 3.2286054167mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
> On Sept. 5, 2016, 11:23 p.m., Jiang Yan Xu wrote: > > src/tests/hierarchical_allocator_tests.cpp, lines 3126-3128 > > <https://reviews.apache.org/r/51028/diff/2/?file=1476333#file1476333line3126> > > > > After reading this comment people could still wonder why this is > > important because we are not measuring how many allocations we did here so > > why does it matter? > > > > I suggest something along the lines of > > > > ``` > > // Wait for the allocation triggered from `addFramework(framework1)` > > // to complete. Otherwise due to a race between > > `addFramework(framework2)` > > // and the next allocation (because it's run asynchronously), > > framework2 > > // may or may not be allocated resources. For simplicity here we give > > // all resources to framework1 as all we wanted to achieve in this > > step > > // is to recover all resources to set up the allocator for the next > > batch > > // allocation. > > ``` Agreed, this is a more thorough explanation. - Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review147697 --- On Aug. 23, 2016, 8:47 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Aug. 23, 2016, 8:47 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Settle the clock > between these events in tests sensitive to this to ensure > expected allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > cbed333f497016fe2811f755028796012b41db77 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
Re: Review Request 51028: Fix tests with rapidly triggered allocations.
- Jacob --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51028/#review147697 --- On Aug. 23, 2016, 8:47 a.m., Jacob Janco wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51028/ > --- > > (Updated Aug. 23, 2016, 8:47 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > --- > > - Per MESOS-3157, if cluster events with the possibility > of triggering an allocation occur rapidly and test > assertions depend on gleaning information from assumed > order, it is likely they will fail due to lack of parity > between event and actual allocation. Settle the clock > between these events in tests sensitive to this to ensure > expected allocations occur. > > > Diffs > - > > src/tests/hierarchical_allocator_tests.cpp > cbed333f497016fe2811f755028796012b41db77 > > Diff: https://reviews.apache.org/r/51028/diff/ > > > Testing > --- > > make check > > > Thanks, > > Jacob Janco > >
