Benjamin Mahler created MESOS-6941:
--------------------------------------
Summary: Add support for batch processing of status updates, to
increase latency / throughput / cluster scalability.
Key: MESOS-6941
URL: https://issues.apache.org/jira/browse/MESOS-6941
Project: Mesos
Issue Type: Improvement
Components: agent, framework api, master
Reporter: Benjamin Mahler
Currently, each task has its own independent status update stream. Within an
individual stream, updates are sent to schedulers in a serial fashion: the
agent will send the N+1th status update only after it receives the scheduler's
acknowledgement for the Nth status update.
This approach slows down throughput substantially and has the potential to
backlog status updates when they occur rapidly. Rather, we should add the
ability for all available updates on a stream to be sent together (there should
probably be a limit to the size of this "batch") so that the scheduler can
process them together without incurring the round trip acknowledgement latency
in between each update.
In addition, there may be cases of updates where the scheduler only wants the
latest information (e.g. download status per MESOS-2256, or possibly health
information). But this should be tackled separately.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
