[
https://issues.apache.org/jira/browse/MESOS-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634214#comment-13634214
]
Benjamin Mahler commented on MESOS-433:
---------------------------------------
This is starting to suggest that mesos is providing at least part of an
observability stack on behalf of executors, something that I'm not convinced is
a good idea, merely in terms of keeping down the number of features we provide.
Presumably, each executor can host an endpoint and the framework will know
about all of the executors' endpoints. Whatever observability stack is used can
then ask each endpoint for the relevant data.
Of course, there are likely many frameworks that will need to implement this,
so it may very well be something we should implement, but I want to make sure
the benefits are clear before adding any complexity:
1. One endpoint per slave, rather than multiple endpoints for the executors.
2. Mesos will store the data on behalf of executors, but executors probably
need to cache this data anyway.
3. ?
Disadvantages:
1. Adding complexity.
2. This exposes a(nother) DOS technique, malicious executors can send large
JSON blobs frequently. The slave may have to manage its memory in order to
mitigate this.
3. ?
----------------------------------------------------------------------------
But, assuming we were to implement such a feature:
This would probably be best as an added call on the ExecutorDriver, allowing
executors to export stats to the slave.
All of the existing Executor interface calls do not return values, and rather
the Executor has to take action by making calls to the driver. Optionally we
can add a call to the Executor interface, asking for json. This would be
beneficial for frameworks that want to be polled for exporting.
Additionally, the slave needs an endpoint for exporting JSON on behalf of the
executors. Currently, the ExecutorDriver (see: exec.cpp) only sends messages to
the slave, so the JSON may have to be passed through the slave for simplicity
initially. We probably want to then create a new libprocess Process to be
responsible for this storing / exposing the JSON for each executor.
This is how I would implement this:
1. Add export(json) to the ExecutorDriver.
2. Send the JSON data to the slave through a new message type.
3. Have the slave relay the data to a new libprocess Process that exports the
data on a /executors/exported.json or the like.
> add exportStats method to executor
> ----------------------------------
>
> Key: MESOS-433
> URL: https://issues.apache.org/jira/browse/MESOS-433
> Project: Mesos
> Issue Type: New Feature
> Components: slave, webui
> Reporter: brian wickman
>
> Now that the slave exports per-executor stats via the UI, the slave should be
> able to periodically request application-specific stats from the executor
> (e.g. zookeeper session id, connection counts, whatever) and export them on
> the executor's behalf. This way we don't need to reinvent any observability
> wheels.
> The interface could either be exportStats(task_id) or just exportStats(), and
> return a JSON object with all stats to export.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
