+Dev
Hey Klaus,
Using Stout's `Optional` to represent the `optional` concept of a message
in protobuf is definitely a step in the right direction.
Regarding your comment in slack yesterday: From my version of the protobuf
generated code there definitely is dynamic allocation even for scalars.
It looks like in our case there is a minimum of 3 dynamic allocations per
Resource object:
> void Resource::SharedDtor() {
> if (name_ != &::google::protobuf::internal::kEmptyString) {
> delete name_;
> }
> if (role_ != _default_role_) {
> delete role_;
> }
> if (this != default_instance_) {
> delete scalar_;
> delete ranges_;
> delete set_;
> delete reservation_;
> delete disk_;
> delete revocable_;
> }
> }
The 2x number I mentioned came from running some of the existing
benchmarks. I didn't explore further because it didn't have as big an
impact as I had hoped. The first battle is simplifying some of the
algorithms in the Sorter / Resources. Once that is done then the resource
arithmetic will be more of a bounding factor.
I agree with Ben that we should focus on writing some basic benchmarks that
represent the common uses of Resources in the allocator. We should scale
these benchmarks to represent some of the more stressful environments that
could occur. For example, had we had such a benchmark, we would have
realized much earlier on that we needed to aggregate only quantities in the
Sorter, and that using the existing form of Resources would have led to a
grinding halt if a reservation were made on every machine.
Is there a regular call that is scheduled to discuss this? I think there
are some other folks also working on benchmarks and interested in the
discussion.
—
*Joris Van Remoortere*
Mesosphere
On Sun, Jul 10, 2016 at 8:50 PM, Klaus Ma <klaus1982...@gmail.com> wrote:
> + more devs :).
>
> ----
> Da (Klaus), Ma (马达) | PMP® | Software Architect
> Platform OpenSource Technology, STG, IBM GCG
> +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me
>
> On Mon, Jul 11, 2016 at 7:43 AM, Klaus Ma <klaus1982...@gmail.com> wrote:
>
>> Hi Joris,
>>
>> I think `Option` is helpful to the performance improvement, it used
>> `placement new` to avoid dynamic allocation. Suppose you're using Option
>> for optional member in protobuf, and using class instance directly
>> (operator=).
>>
>> I'm adding some benchmark for `Resources`, especially for the `Resources`
>> with Rang, DiskInfo and ReservationInfo
>>
>> Draft PR for Benchmark of Resources:
>> https://github.com/k82cn/mesos/commit/09ca215cb37b1f89eb7d622228a8cf2249eb641c
>>
>>
>> ----
>> Da (Klaus), Ma (马达) | PMP® | Software Architect
>> Platform OpenSource Technology, STG, IBM GCG
>> +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me
>>
>
>
