[
https://issues.apache.org/jira/browse/MESOS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878882#comment-16878882
]
Benjamin Mahler commented on MESOS-9755:
----------------------------------------
For posterity, it looks like there is a performance regression in the v0 API
when upgrading to protobuf 3.7.1:
Master:
{noformat}
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/0
Test setup: 1000 agents with a total of 10000 running tasks and 10000 completed
tasks
v0 '/state' response took 177.001464ms
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/0
(4593 ms)
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/1
Test setup: 10000 agents with a total of 100000 running tasks and 100000
completed tasks
v0 '/state' response took 1.802505171secs
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/1
(51571 ms)
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/2
Test setup: 20000 agents with a total of 200000 running tasks and 200000
completed tasks
v0 '/state' response took 3.164482263secs
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/2
(104737 ms)
{noformat}
After upgrading to 3.7.1:
{noformat}
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/0
Test setup: 1000 agents with a total of 10000 running tasks and 10000 completed
tasks
v0 '/state' response took 253.753947ms
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/0
(6107 ms)
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/1
Test setup: 10000 agents with a total of 100000 running tasks and 100000
completed tasks
v0 '/state' response took 2.118297secs
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/1
(58902 ms)
[ RUN ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/2
Test setup: 20000 agents with a total of 200000 running tasks and 200000
completed tasks
v0 '/state' response took 4.150050151secs
[ OK ]
AgentFrameworkTaskCountContentType/MasterStateQuery_BENCHMARK_Test.GetState/2
(116661 ms)
{noformat}
It appears to be due to a performance regression in the reflection code in
protobuf. We may want to investigate further with the protobuf maintainers
and/or investigate using the built in json conversion support rather than our
reflection based implementation.
> Upgrade bundled protobuf to 3.7.x.
> ----------------------------------
>
> Key: MESOS-9755
> URL: https://issues.apache.org/jira/browse/MESOS-9755
> Project: Mesos
> Issue Type: Wish
> Reporter: Kaiwalya Joshi
> Priority: Major
> Labels: foundations, integration, protobuf
>
> We're noticing the following warning emitted by the JVM on JDK9+ for Google
> Protobuf _v3.5.0_
> {code}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil
> (file:/home/kjoshi/.gradle/caches/modules-2/files-2.1/com.google.protobuf/protobuf-java/3.5.0/200fb936907fbab5e521d148026f6033d4aa539e/protobuf-java-3.5.0.jar)
> to field java.nio.Buffer.address
> WARNING: Please consider reporting this to the maintainers of
> com.google.protobuf.UnsafeUtil
> {code}
> This warning is fixed in ProtoBuf versions [_v3.7.0_ and
> above|https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.0].
> As the current access warning can turn into an access violation in later
> versions of the JDK, we're requesting Mesos to update to a version of
> ProtoBuf that incorporates the needed fixes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
