This is great, and will help a lot! I do wish there was a way to store structured data rather than just a string, but this will do to start :)
On Dec 18, 2014, at 2:22 PM, Benjamin Mahler <[email protected]> wrote: > Hey Steven, > > We introduced a field on the ExecutorInfo called "source" [1] (which we > didn't document well, sorry about that). This field should also be introduced > on the TaskInfo [2] for those who just use the built-in CommandExecutor. > > When launching an executor or a task, you can specify a "source" that allows > you to identify the executor for monitoring / reporting purposes. This is how > we do our "container" monitoring at Twitter: > > (1) Frameworks set "source" to a meaningful string: e.g. > "prod.ads-team.ads-server.0". (environment, team, service, instance). > (2) An internal system collects minutely data from /monitor/statistics.json > on the slave. > (3) The metrics include all those you listed above (disk utilization is > coming). Once these are stored in an internal TSDB, dashboards, queries, and > reports can be constructed for the relevant services, teams, etc. > > I hope this helps, IMHO this is probably the simplest path forward for you, > as you don't need to have any pluggable functionality built-in to mesos to > get what you need. > > Ben > > [1] https://github.com/apache/mesos/blob/0.21.0/include/mesos/mesos.proto#L283 > [2] https://issues.apache.org/jira/browse/MESOS-486 > > On Thu, Dec 18, 2014 at 11:56 AM, Steven Schlansker > <[email protected]> wrote: > I am running a corporate Mesos cluster, shared by a number of teams and > projects. > We are looking to get some insight into our usage of precious computing > resources. For example, I'd like to be able to present a report breaking > down CPU-hour and RAM GB-hour utilization by service, team, or other relevant > grouping. > > How I'd imagine this works: > > * Collect Mesos statistics per task (allocated CPU, CPU utilization, > allocated memory, memory utilization, disk utilization) periodically (say, > once a minute) > * Collect task metadata from a pluggable source (mapping from Mesos task to > service name, team name, any other metadata you wish to use to group tasks) > * Generate dashboard / reports by aggregating task data over axes provided by > metadata input > > Has anyone started on such a project? > > Thanks, > Steven >
