> On Dec. 9, 2014, 9:45 p.m., Jie Yu wrote: > > The performance improvement is significant. Using the benchmark in > > https://reviews.apache.org/r/28871 > > > > Before: > > ``` > > [jyu@smfd-aki-27-sr1 build]$ 3rdparty/libprocess/benchmarks > > --gtest_filter=*Link* > > Note: Google Test filter = *Link* > > [==========] Running 1 test from 1 test case. > > [----------] Global test environment set-up. > > [----------] 1 test from Process > > [ RUN ] Process.Process_BENCHMARK_LargeNumberOfLinks > > Elapsed: 11.616946676secs > > [ OK ] Process.Process_BENCHMARK_LargeNumberOfLinks (18242 ms) > > [----------] 1 test from Process (18244 ms total) > > > > [----------] Global test environment tear-down > > [==========] 1 test from 1 test case ran. (18244 ms total) > > ``` > > > > After: > > ``` > > [jyu@smfd-aki-27-sr1 build]$ 3rdparty/libprocess/benchmarks > > --gtest_filter=*Link* > > Note: Google Test filter = *Link* > > [==========] Running 1 test from 1 test case. > > [----------] Global test environment set-up. > > [----------] 1 test from Process > > [ RUN ] Process.Process_BENCHMARK_LargeNumberOfLinks > > Elapsed: 500.926362ms > > [ OK ] Process.Process_BENCHMARK_LargeNumberOfLinks (1391 ms) > > [----------] 1 test from Process (1393 ms total) > > > > [----------] Global test environment tear-down > > [==========] 1 test from 1 test case ran. (1393 ms total) > > [ PASSED ] 1 test. > > ``` > > Dominic Hamon wrote: > +1 > > or rather, x20!
Increased your benchmark to 20,000 links to simulate a production master a bit more accurately: 54 seconds (old code) vs 680 milliseconds (w/ this patch). Looks like the old code scales quite poorly. - Ben ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28838/#review64442 ----------------------------------------------------------- On Dec. 9, 2014, 9:07 p.m., Ben Mahler wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/28838/ > ----------------------------------------------------------- > > (Updated Dec. 9, 2014, 9:07 p.m.) > > > Review request for mesos, Benjamin Hindman, Jie Yu, and Joris Van Remoortere. > > > Bugs: MESOS-2182 > https://issues.apache.org/jira/browse/MESOS-2182 > > > Repository: mesos-git > > > Description > ------- > > See MESOS-2182. > > The iteration over the links is expensive _and_ occurs within the > SocketManager's critical section, which we think is having some bad effects > blocking other calls (see the comments in the ticket). > > This change updates the socket manager to keep a bi-directional mapping > between the "linkers" and the "linkees", which means that we now only look at > the relevant information when a node/process exits. > > Note that I did double lookups on the maps, this was because we do this > heavily in libprocess already. I had originally written out the code using > .find() to avoid the double lookups, but it became next to impossible to > read. Let's micro-optimize later, this is a major improvement as it is. :) > > I tried to keep the additional complexity in check, let me know if there are > any suggestions to make it easier! > > > Diffs > ----- > > 3rdparty/libprocess/src/process.cpp > b87ac2206548815bc992c955252567c131fe6a47 > > Diff: https://reviews.apache.org/r/28838/diff/ > > > Testing > ------- > > make check > > Manually started a master and slave across machines, to ensure exit > notifications were sent correctly. > > > Thanks, > > Ben Mahler > >
