[
https://issues.apache.org/jira/browse/MESOS-10026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Mahler reassigned MESOS-10026:
---------------------------------------
Assignee: Benjamin Mahler
> Improve v1 operator API read performance.
> -----------------------------------------
>
> Key: MESOS-10026
> URL: https://issues.apache.org/jira/browse/MESOS-10026
> Project: Mesos
> Issue Type: Improvement
> Components: HTTP API
> Reporter: Benjamin Mahler
> Assignee: Benjamin Mahler
> Priority: Major
> Labels: foundations
>
> Currently, the v1 operator API has poor performance relative to the v0 json
> API. The following initial numbers were provided by [~Will Mahler] from our
> state serving benchmark:
>
> |OPTIMIZED - Master (baseline)| | | | |
> |Test setup|1000 agents with a total of 10000 running tasks and 10000
> completed tasks|10000 agents with a total of 100000 running tasks and 100000
> completed tasks|20000 agents with a total of 200000 running tasks and 200000
> completed tasks|40000 agents with a total of 400000 running tasks and 400000
> completed tasks|
> |v0 'state' response|0.17|1.66|8.96|12.42|
> |v1 x-protobuf|0.35|3.21|9.47|19.09|
> |v1 json|0.45|4.72|10.81|31.43|
> There is quite a lot of variance, but v1 protobuf consistently slower than v0
> (sometimes significantly so) and v1 json is consistently slower than v1
> protobuf (sometimes significantly so).
> The reason that the v1 operator API is slower is that it does the following:
> (1) Construct temporary unversioned state response object by copying
> in-memory un-versioned state into overall response object. (expensive!)
> (2) Evolve it to v1: serialize, de-serialize into v1 overall state object.
> (expensive!)
> (3) Serialize the overall v1 state object to protobuf or json.
> (4) Destruct the temporaries (expensive! but is done after response starts
> serving)
> On the other hand, the v0 jsonify approach does the following:
> (1) Serialize the in-memory unversioned state into json, by traversing state
> and accumulating the overall serialized json.
> This means that v1 has substantial overhead vs v0, and we need to remove it
> to bring v1 on-par or better than v0. v1 should serialize directly to json
> (straightforward with jsonify) or protobuf (this can be done via a
> io::CodedOutputStream).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
