----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67871/ -----------------------------------------------------------
(Updated July 12, 2018, 9:39 p.m.) Review request for mesos, Benjamin Mahler, Gastón Kleiman, and James Peach. Repository: mesos Description ------- Profiling of metrics generation revealed a large amount of time spent in map operations. This patch does three things to mitigate this: * Stores the metrics as an ordered map so that we only pay the price of sorting when the metric is first added. * Makes use of vectors instead of maps for intermediate objects, which eliminates the need for another intermediate object. * Hints when inserting into the returned map, reducing the cost of insertion into that ordered container. Diffs (updated) ----- 3rdparty/libprocess/include/process/metrics/metrics.hpp f9b72029b2c85826c91b1d7656b0af94dc87010c 3rdparty/libprocess/src/metrics/metrics.cpp 4883c9acaa0cc568e27944661a8208f7b2a356a1 Diff: https://reviews.apache.org/r/67871/diff/3/ Changes: https://reviews.apache.org/r/67871/diff/2-3/ Testing ------- WITH per-framework metrics, BEFORE optimizations: ``` [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/0 Test setup: 1 agents with a total of 105 frameworks unversioned /metrics/snapshot' response took 157.1449ms v1 'master::call::GetMetrics' application/x-protobuf response took 152.599692ms v1 'master::call::GetMetrics' application/json response took 198.918334ms [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/0 (835 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/1 Test setup: 1 agents with a total of 1005 frameworks unversioned /metrics/snapshot' response took 1.319444199secs v1 'master::call::GetMetrics' application/x-protobuf response took 1.257644596secs v1 'master::call::GetMetrics' application/json response took 1.527225235secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/1 (6553 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/2 Test setup: 1 agents with a total of 10005 frameworks unversioned /metrics/snapshot' response took 15.479365874secs v1 'master::call::GetMetrics' application/x-protobuf response took 14.542866983secs v1 'master::call::GetMetrics' application/json response took 18.05492789secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/2 (75455 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/3 Test setup: 1 agents with a total of 20005 frameworks unversioned /metrics/snapshot' response took 31.908301664secs v1 'master::call::GetMetrics' application/x-protobuf response took 32.128689785secs v1 'master::call::GetMetrics' application/json response took 33.669376185secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/3 (150440 ms) ``` WITH per-framework metrics, AFTER optimizations: ``` [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/0 Test setup: 1 agents with a total of 105 frameworks unversioned /metrics/snapshot' response took 104.577895ms v1 'master::call::GetMetrics' application/x-protobuf response took 74.262533ms v1 'master::call::GetMetrics' application/json response took 100.218618ms [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/0 (562 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/1 Test setup: 1 agents with a total of 1005 frameworks unversioned /metrics/snapshot' response took 921.175877ms v1 'master::call::GetMetrics' application/x-protobuf response took 780.277639ms v1 'master::call::GetMetrics' application/json response took 1.168651111secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/1 (5424 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/2 Test setup: 1 agents with a total of 10005 frameworks unversioned /metrics/snapshot' response took 10.2413387secs v1 'master::call::GetMetrics' application/x-protobuf response took 9.407992945secs v1 'master::call::GetMetrics' application/json response took 10.582584848secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/2 (57206 ms) [ RUN ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/3 Test setup: 1 agents with a total of 20005 frameworks unversioned /metrics/snapshot' response took 19.930542079secs v1 'master::call::GetMetrics' application/x-protobuf response took 20.318739763secs v1 'master::call::GetMetrics' application/json response took 22.853630899secs [ OK ] AgentFrameworkTaskCountContentType/MasterMetricsQuery_BENCHMARK_Test.GetMetrics/3 (116363 ms) ``` Thanks, Greg Mann
