[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393410#comment-15393410
]
Guangya Liu commented on MESOS-5700:
Did some test for how does {{addable}} and {{subtractable}} contribute to
resources benchmark test, the result is that {{those two validations does not
cost much time and we can ignore it}}. cc [~bmahler] [~klaus1982]
Test steps are as following:
1) Checkout two source code copies: mesos-1 and mesos-2, apply patch
https://reviews.apache.org/r/50380/ for both copies.
2) Update code in mesos-1 by removing both {{addable}} and {{subtractable}} for
resources {{+=}} and {{-=}}. Code diff is as following:
{code}
diff --git a/src/common/resources.cpp b/src/common/resources.cpp
index 3dbff24..d770e98 100644
--- a/src/common/resources.cpp
+++ b/src/common/resources.cpp
@@ -227,6 +227,7 @@ bool operator!=(const Resource& left, const Resource& right)
namespace internal {
+#if 0
// Tests if we can add two Resource objects together resulting in one
// valid Resource object. For example, two Resource objects with
// different name, type or role are not addable.
@@ -277,6 +278,7 @@ static bool addable(const Resource& left, const Resource&
right)
return true;
}
+#endif
// Tests if we can subtract "right" from "left" resulting in one valid
@@ -1381,11 +1383,9 @@ void Resources::add(const Resource& that)
bool found = false;
foreach (Resource& resource, resources) {
-if (internal::addable(resource, that)) {
resource += that;
found = true;
break;
-}
}
// Cannot be combined with any existing Resource object.
@@ -1439,7 +1439,6 @@ void Resources::subtract(const Resource& that)
for (int i = 0; i < resources.size(); i++) {
Resource* resource = resources.Mutable(i);
-if (internal::subtractable(*resource, that)) {
*resource -= that;
// Remove the resource if it becomes invalid or zero. We need
@@ -1455,7 +1454,6 @@ void Resources::subtract(const Resource& that)
}
{code}
3) Build those two copies and run benchmark test
{{ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2}}.
Test result without validation for both {{addable}} and {{subtractable}}
{code}
[==] Running 1 test from 1 test case.
[--] Global test environment set-up.
[--] 1 test from ResourcesOperators/Resources_BENCHMARK_Test
[ RUN ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2
Took 2.833678secs to perform 1000 'total += r' operations on ports(*):[1-2,
4-5, 7-8, 10-11, 13-14, 16-17, 1...
Took 3.656634secs to perform 1000 'total -= r' operations on ports(*):[1-2,
4-5, 7-8, 10-11, 13-14, 16-17, 1...
Took 3.012337secs to perform 1000 'total = total + r' operations on
ports(*):[1-2, 4-5, 7-8, 10-11, 13-14, 16-17, 1...
Took 3.650337secs to perform 1000 'total = total - r' operations on
ports(*):[1-2, 4-5, 7-8, 10-11, 13-14, 16-17, 1...
[ OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2 (13155 ms)
[--] 1 test from ResourcesOperators/Resources_BENCHMARK_Test (13155 ms
total)
[--] Global test environment tear-down
[==] 1 test from 1 test case ran. (13174 ms total)
[ PASSED ] 1 test.
{code}
Test result with validation for both {{addable}} and {{subtractable}}
{code}
[==] Running 1 test from 1 test case.
[--] Global test environment set-up.
[--] 1 test from ResourcesOperators/Resources_BENCHMARK_Test
[ RUN ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2
Took 2.707476secs to perform 1000 'total += r' operations on ports(*):[1-2,
4-5, 7-8, 10-11, 13-14, 16-17, 1...
Took 3.49798secs to perform 1000 'total -= r' operations on ports(*):[1-2, 4-5,
7-8, 10-11, 13-14, 16-17, 1...
Took 2.911038secs to perform 1000 'total = total + r' operations on
ports(*):[1-2, 4-5, 7-8, 10-11, 13-14, 16-17, 1...
Took 3.692435secs to perform 1000 'total = total - r' operations on
ports(*):[1-2, 4-5, 7-8, 10-11, 13-14, 16-17, 1...
[ OK ] ResourcesOperators/Resources_BENCHMARK_Test.Arithmetic/2 (12811 ms)
[--] 1 test from ResourcesOperators/Resources_BENCHMARK_Test (12811 ms
total)
[--] Global test environment tear-down
[==] 1 test from 1 test case ran. (12830 ms total)
[ PASSED ] 1 test.
{code}
Please refer to
https://docs.google.com/document/d/1D5qqkEh28vnS-2j3F1K8liYS8ThtSjeLJ4AvogIoxjk/edit?ts=57971af2#
for more detail of the diagram of {{valgrind --tool=callgrind}}.
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
> Attachments: hashmap.diff, name_roleId.diff, port.perf.log,
> reservation.perf.log
>
>
> Add
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381362#comment-15381362
]
Klaus Ma commented on MESOS-5700:
-
Did more test based on the Resources benchmark:
1. make Resource's name & role to be enum/int will improve 400ms per 10
operator on 1000 resource.
2. using hashmap in Resources will improve 2 s per 10 operator on 4000
resource; it should be more improved if using name + role as key. The overhead
of hash map is 70ms per 5000 operators on 4 resources.
Please refer to the attachment for the code diff I used.
*Test result for Resource's name & role*:
Using name() & role() in {{addable()}}
{code}
Took 577.479396ms to perform 10 'total += r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 647.031984ms to perform 10 'total -= r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 620.093549ms to perform 10 'total = total + r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 693.93056ms to perform 10 'total = total - r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
{code}
without name() but with role() in {{addable()}}
{code}
Took 279.432157ms to perform 10 'total += r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 647.879612ms to perform 10 'total -= r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 323.552389ms to perform 10 'total = total + r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
Took 682.131788ms to perform 10 'total = total - r' operations on cpus(role_0,
principal_0, {key_0: value_0}):1; ...
{code}
without name() and with role_id() in {{addable()}}
{code}
Took 175.373644ms to perform 10 'total += r' operations on cpus(0, principal_0,
{key_0: value_0}):1; cpus(...
Took 598.86704ms to perform 10 'total -= r' operations on cpus(0, principal_0,
{key_0: value_0}):1; cpus(...
Took 222.053999ms to perform 10 'total = total + r' operations on cpus(0,
principal_0, {key_0: value_0}):1; cpus(...
Took 616.455333ms to perform 10 'total = total - r' operations on cpus(0,
principal_0, {key_0: value_0}):1; cpus(...
{code}
*Test result for hash map for Resources*:
With cpus/gpus/disk
{code}
Took 4.396598secs to perform 10 'total += r' operations on cpus(0, principal_0,
{key_0: value_0}):1; gpus(...
Took 4.73346secs to perform 10 'total -= r' operations on cpus(0, principal_0,
{key_0: value_0}):1; gpus(...
Took 4.437688secs to perform 10 'total = total + r' operations on cpus(0,
principal_0, {key_0: value_0}):1; gpus(...
Took 4.940758secs to perform 10 'total = total - r' operations on cpus(0,
principal_0, {key_0: value_0}):1; gpus(...
{code}
Using map for operator+=
{code}
Took 2.469885secs to perform 10 'total += r' operations on cpus(0, principal_0,
{key_0: value_0}):1; gpus(...
Took 4.806674secs to perform 10 'total -= r' operations on cpus(0, principal_0,
{key_0: value_0}):1; gpus(...
Took 3.0442secs to perform 10 'total = total + r' operations on cpus(0,
principal_0, {key_0: value_0}):1; gpus(...
Took 3.278178secs to perform 10 'total = total - r' operations on cpus(0,
principal_0, {key_0: value_0}):1; gpus(...
{code}
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
> Attachments: port.perf.log, reservation.perf.log
>
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380797#comment-15380797
]
Klaus Ma commented on MESOS-5700:
-
Just did a profiling for Resources class based on the benchmark, please refer
to the attachment for the detail. Here's suggestion to improve the performance:
1. In reservation.perf.log, it shows {{operator!=(string, string)}} in
{{addable/subtractable}} is the heavy function; so I'm thinking to
* Make name of Resource to be enum instread of string (as we discussed in
the doc)
* Introduce roleId (uint64_t), similar to uid in Linux; {{roleId}} is used
within Mesos master/agent
2. In port.perf.log, it shows {{ranges validation}} is the heavy function; no
conclusion for now, but it's better to refine {{Resource::validate}}:
* role validation is not necessary in every operator/loop
* ranges validation maybe not necessary in every operator/loop,
{{coalesce}} may handle it (investigating)
cc [~bmahler]/[~gyliu]/[~alexr]
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368666#comment-15368666
]
Guangya Liu commented on MESOS-5700:
Based on investigation from [~jvanremoortere] and [~mcypark] , the founding is
that (1) copying of the protobufs was expensive (2) looping over and checking
.name() equality was expensive, for example. We may need to think more use
cases related to {{Resource}} and translate those to benchmark test.
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15356615#comment-15356615
]
Klaus Ma commented on MESOS-5700:
-
Here's benchmark for {{operator+}} & {{operator+=}}
{code}
[ RUN ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/5
Added 100 resources (cpus:1) in 437384us
[ OK ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/5 (438 ms)
[ RUN ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/11
Added 100 resources (cpus:1;mem:2) in 826587us
[ OK ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/11 (826
ms)
[ RUN ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/17
Added 100 resources (cpus:1;ports:[1-100]) in 1.944934secs
[ OK ]
ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_AddAndAssign/17 (1945
ms)
[ RUN ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/5
Added 100 resources (cpus:1) in 1.368948secs
[ OK ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/5
(1369 ms)
[ RUN ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/11
Added 100 resources (cpus:1;mem:2) in 2.734078secs
[ OK ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/11
(2734 ms)
[ RUN ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/17
Added 100 resources (cpus:1;ports:[1-100]) in 4.410165secs
[ OK ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/17
(4410 ms)
{code}
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15356374#comment-15356374
]
Klaus Ma commented on MESOS-5700:
-
Will add more benchmark for {{operator+(...)}}, according to the output of
{{calgrind}}, CopyFrom is the heavy operators.
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5700) Benchmark for Resource class (protobuf vs. C++)
[
https://issues.apache.org/jira/browse/MESOS-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355336#comment-15355336
]
Klaus Ma commented on MESOS-5700:
-
Keep add the same resource for 100 times:
{code}
[ RUN ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/17
Added 100 resources (cpus:1;ports:[1-100]) in 1.895914secs
[ OK ] ResourcesOperatorCount/Resources_BENCHMARK_Test.Operator_Add/17
(1896 ms)
{code}
> Benchmark for Resource class (protobuf vs. C++)
> ---
>
> Key: MESOS-5700
> URL: https://issues.apache.org/jira/browse/MESOS-5700
> Project: Mesos
> Issue Type: Bug
>Reporter: Klaus Ma
>Assignee: Klaus Ma
>
> Add benchmark of Resource class for Allocation Performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
