[
https://issues.apache.org/jira/browse/MESOS-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624276#comment-16624276
]
Meng Zhu commented on MESOS-6765:
---------------------------------
{noformat}
commit fa7f659b1952344cf52c725ccdf8f86d7a01296b
Author: Meng Zhu
Date: Thu Sep 20 13:20:06 2018 -0700
Optimized `class Resources` with copy-on-write.
This patch lets `class Resources` only store shared
pointers to the underlying resource objects, so that
read-only filter operations such as `reserved()`,
`unreserved()` and etc. can avoid making copies of
the whole resource objects. Instead, only shared pointers
are copied.
In write operations, we check if there are more than one
references to the resource object. If so, a copy is made
for safe mutation without affecting owners.
To maintain the usage abstraction that `class Resources`
still holds resource objects, we utilize
`boost::indirect_iterator` iterator adapter to deference
the shared pointers as we iterate.
Review: https://reviews.apache.org/r/68490/
{noformat}
{noformat}
commit 4ffd1099ca8f3a747de9a5282c4cac791113c5cc (HEAD -> master, apache/master)
Author: Meng Zhu [email protected]
Date: Thu Sep 20 14:03:00 2018 -0700
Mitigated accidental mutation of `Resources::resources`.
Due to the copy-on-write optimization (MESOS-6765), one needs to
check the `use_count` of `Resource_` before mutating. Currently,
there is no mechanism to enforce this. As a short-term mitigation
measure, we rename `resources` to
`resourcesNoMutationWithoutExclusiveOwnership` and typedef its item
type to `Resource_UnSafe`
to alert people about obtaining an exclusive ownership before mutating
the `Resource_` objects.
{noformat}
> Consider making the Resources wrapper "copy-on-write" to improve performance.
> -----------------------------------------------------------------------------
>
> Key: MESOS-6765
> URL: https://issues.apache.org/jira/browse/MESOS-6765
> Project: Mesos
> Issue Type: Improvement
> Reporter: Benjamin Mahler
> Assignee: Meng Zhu
> Priority: Major
> Labels: mesosphere, performance
> Fix For: 1.8.0
>
>
> Resources currently directly stores the underlying resource objects:
> {code}
> class Resources
> {
> ...
> std::vector<Resource_> resources;
> };
> {code}
> What this means is that copying of Resources (which occurs frequently) is
> expensive since copying a {{Resource}} object is relatively heavy-weight.
> One strategy, in MESOS-4770, is to avoid protobuf in favor of C++ types (i.e.
> replace {{Value::Scalar}}, {{Value::Set}}, and {{Value::Ranges}} with C++
> equivalents). However, metadata like reservations, disk info, etc, is still
> fairly expensive to copy even if avoiding protobufs.
> An approach to reduce copying would be to only copy the resource objects upon
> writing, when there are multiple references to the resource object. If there
> is a single reference to the resource object we could safely mutate it
> without copying. E.g.
> {code}
> class Resource
> {
> ...
> std::vector<shared_ptr<Resource_>> resources;
> };
> // Mutation function:
> void Resources::mutate(size_t index)
> {
> // Copy if there are multiple references.
> if (resources[i].use_count() > 1) {
> resources[i] = copy(resources[i]);
> }
> // Mutate safely.
> resources[i].some_mutation();
> }
> {code}
> On the other hand, this introduces a additional level of pointer chasing. So
> we would need to weigh the approaches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
